Photogrammetry


[WIP; page under construction]

Introduction

The aim of this document is to introduce the reader to photogrammetry. To that end, both a formal conceptual definition and a practical description of the production process are provided. This text finishes with a discussion of additional practical topics that will hopefully guide or at least assist the reader when they embark upon their own photogrammetry project.

The author of this document, Daniel Muirhead, has (at the time of writing) captured more than 250,000 photographs for use in photogrammetry projects, in the process logging more than 5,500 hours in a variety of photogrammetry softwares.


Definition

In this context, by “photogrammetry” is meant the photo capture of a real world subject and the subsequent use of these photos as inputs in a computer program (“photogrammetry software”) which automatically reconstructs that subject as a virtual 3D model.


Production

  • Equipment
  • Workflow

Equipment

Photogrammetry involves 4 pieces of equipment:

  1. capture device;
  2. conveyance of capture device;
  3. photogrammetry software;
  4. computer (for running the software).

Capture device, examples:

  • any (digital) camera which is capable of capturing (digital) photos;
  • any (digital) camcorder which is capable of capturing (digital) video, this video can be decomposed into a sequence of stills which then serve as ‘photos’;
  • in theory, any device which is consistent in the way it captures/projects the 3D world onto a 2D image.

Conveyance of capture device, examples:

  • the capture device is attached to a tripod with adjustable elevation while the subject is mounted on a rotating turntable;
  • the capture device is held by (or otherwise attached to) the photographer as they traverse around the subject;
  • the capture device is mounted on a vehicle (e.g. atop a ground-based vehicle or underside an aerial-based vehicle) which, being manually/remotely/autonomously controlled, traverses around the subject.

Photogrammetry software, examples:

  • a variety of commercial softwares are available, however this document is intended as an unbiased overview of the photogrammetry production process, so while in reality I do use commercial photogrammetry software and applaud some companies and hold others in disdain, in the interests of impartiality I do not make any recommendations here, and instead advise the reader to perform their own research and experimentation and come to their own conclusions;
  • a number of Free and Open Source Software (FOSS) photogrammetry applications are available, these include COLMAP and Meshroom.

Computer, examples:

  • if possible, it is highly advisable to run the photogrammetry software on a computer with a graphics card (GPU) which is CUDA-enabled, because almost all the photogrammetry software that this author has experience of performs significantly faster on a computer with a CUDA-enabled GPU compared to that same computer when only utilising its central processor (CPU);
  • other factors that can affect the photogrammetry software processing speed are the specification of the CPU, the amount of RAM available, and whether the project files (and the software itself and its working/temp file cache) are located on a solid state drive (SSD) instead of a hard disk drive (HDD), in addition to the speed of the connection between the storage (SDD/HDD) and the motherboard;
  • again, no specific computer specifications are provided here, instead the reader is recommended to perform their own research, this could involve directly contacting the software developers to receive advice regarding recommended computer specification per specified budget.

Workflow

The process is linear and can be summarised in terms of these successive stages: 1) Capture > 2) Photogrammetry > 3) Digital Asset Generation. Each stage generates its own product, these are: 1) Photos; 2) Virtual 3D (photogrammetry) Model; 3) Further/Derived Digital Assets.

1) Capture/Photos

The real world subject, to be captured, is a three-dimensional volume that is covered by a surface. If, for the sake of description, we visualise the subject’s surface divided into many smaller equal-area patches, then the idea during capture is to photograph each of these patches from a diversity of angles.

Diverse photos of a subject are distinguishable by virtue of variation in their location (position in 3D space quantified as x/y/z coordinates) and/or their orientation (yaw/pitch/roll of capture device). The path that the capture device follows, through successive capture locations and/or orientations, can be described as the capture trajectory.

The complexity of the capture trajectory is co-variant with the complexity of the subject’s surface geometry. That is, subjects with which are simple in shape will (unsurprisingly) be easier to capture than subjects with a complex shape. Subjects with more nooks and crannies, extrusions and partially occluded areas, will require more effort and time to perform a comprehensive capture. Before engaging in capture, if the photographer knows the approximate geometry of the subject, then it should be possible in theory to calculate in advance an optimal capture trajectory.

The overall scale of the subject (e.g. very large subjects), by comparison, has no direct relevance to the capture trajectory, provided the photographer has access to suitable means for conveying the capture device. For example, the capture trajectory for a miniature scale model of a town, and the capture trajectory for its full-size real world equivalent, would in theory be identical (once the scale/size of the two subjects was accounted for). The only difference would be in the mode of conveyance – handheld (or tripod/turntable) in the case of the miniature scale model, and vehicle-mounted in the case of the full size town.

2) Photogrammetry/Virtual 3D (photogrammetry) Model

The photogrammetry process is managed by the photogrammetry software and is mostly automated, involving user input only when: adding photos as inputs; tweaking parameters/settings depending on desired output; and (optionally) performing quality assurance checks mid-production. However the actual activity of reconstructing the virtual 3D model from the photos is performed entirely by the software in the sense that it involves no manual effort or decision-making on the part of the user.

The automated process performed by the photogrammetry software has several successive, discrete stages.


Practical considerations

  • Factors affecting capture (surface, illumination)
  • Recommended capture trajectories

Factors affecting capture (surface, illumination)

A photograph captures the interaction between a source of illumination and a surface, or more usually between diverse sources of illumination and diverse surfaces.

[notes]

Note on hardware cost:

Photogrammetry is a technical (as opposed to artistic*) process which means that all other factors being equal, the quality of the output is determined by the quality (read: cost, expensiveness) of the equipment used during the production process.

Annoyingly this means that photogrammetry is a problem that can be solved by throwing a lot of money at it, so if your financial situation is that of a starving artist then you will unfortunately be at a slight disadvantage as compared to your wealthier competitors. The most important piece of equipment is the computer with CUDA-enabled graphics card, thus as long as you or someone you know already has access to e.g. a gaming PC with an Nvidia graphics card then you are 90% there. The advanced state of photogrammetry software (compared to several years back) means that photo capture can be performed with any cheap digital camera, e.g. a ‘compact’ camera, ‘action camcorder’, cheap smartphone camera, or the camera module of a Raspberry Pi. Similarly the capabilities of free and/or freeware photogrammetry software are fine for processing any project with 50 inputs or less.

Needless to say, access to fancier hardware will make your life easier. However it is possible to perform photogrammetry with a modest setup.

*Photogrammetry outputs may be used for art but the photogrammetry process itself is technical, not artistic – its success depends on the quality of the equipment, and to a degree the competence of the technician, but this is a different type of labour to the manual skill exerted by the artist.

%d bloggers like this: