In this video tutorial, we are focusing on 3D photography or photogrammetry. We will delve into how it works, a bit of its history, show some practical examples and walk us through the process. Along the way, we will look at some of the gotchas and the essentials that you will need to get started. Part 1 of 2.
Archival 3D Photography
3D photography is a thing now. Drones, 3D printers, Google Cardboard and Oculus Rift… we’re seeing applications for 3D imaging everyday. When it comes to producing high quality 3D data from photos, there’s some stuff you’ll want to know to do it right, do it fast, and be able to do it over and over again. This series is all about that.
In part I, I’m going to be focusing on 3D imaging or photogrammetry. I will delve into how it works, a bit of its history, show some practical examples and walk us through the process. Along the way, we will look at some of the gotchas and the essentials that you will need to get started, which are surprisingly minimal (seriously). We’ll end with where to go from here, and some hints at the upcoming tutorials in the series.
Photogrammetry? What is it good for?
Photogrammetry is the science of making measurements from photographs, or making photographs measurable. Common outputs from photogrammetry include:
- Orthophotos. An orthorectified image is geometrically corrected so that the scale is uniform: the photo has the same lack of distortion as a map. A digital orthophoto is created from several photographs mosaicked to form the final image.
- 3D models. Using the same photo series, it’s possible to create a complete 3D model of an object, place, or even a person.
Digital Elevation Model. A digital elevation model (DEM) is a digital model or 3D representation of a terrain’s surface, created from terrain elevation data.
- Animation. Videos can be made directly from interacting with the models in a variety of software applications.
We’re going to take a look at some amazing advances in photography and software that makes creating measurable photos easy and incredibly useful. Once you try it, you won’t be able to stop!
Author: Michael Ashley
In this tutorial, I’m going to be focusing on 3D imaging or photogrammetry, I will delve into how it works, a bit of its history, show some practical examples and walk us through the process. Along the way, we will look at some of the gotchas and the essentials that you will need to get started, which are surprisingly minimal. Finally, I will discuss our born archival imaging workflow that will help you get fantastic results and give you some tips on where to go from here. The Center for Digital Archaeology is a 501c(3) non profit California organization dedicated to making it as easy as possible for individuals and organizations to fully realize the potential of digital technology with the least reliance on experts as possible. We have events and training opportunities going on all the time, so I would encourage you to check out our website or subscribe to our mailing list to keep informed.
Let’s jump right in by taking a look at some examples. These are objects, or rather, these are three-dimensional representations of objects that are very well suited to 3D archival imaging techniques. Depending on the object, there will be different requirements for the environment, the amount of photos that you’ll need to take, and the way in which these photos are taken. But all of it is learnable and thanks to some remarkable advances in just the last few years, it is also very affordable for the first time in history. This is Sven Haakanson, now at the Burke Museum, described at his SHN workshop how he used photogrammetry of boat models found in Russia to help build a full scale replica and bring it back to Kodiak Island for the first time. It was a fantastic community event. He learned this technique himself and produced something that is truly remarkable.
Until recently, a lot of what we thought of we spoke of photogrammetry were two-dimensional aerial photographs. But it has now become rather common, especially in archaeology and cultural heritage, to document places using what is called close range photogrammetry, by either using a pole or monopod, flying a drone or kite. We can bring the resolution even closer to reveal tremendous details when we get even closer to the ground. Or even below it. This 3D model is even more remarkable because it is embedded in a simple Adobe PDF document. I have selected a 3D measurement tool, and as you can see, I can actually measure this pipe right inside of the PDF. Generating two-dimensional, but precisely measured orthographic photographs is very useful for a variety of purposes, including GIS, or geographical information systems.
Here, we have placed the orthophotos generated from our photogrammetry technique directly into our GIS. Because these 3D photographs have real world coordinate information inside of them, they slip right into the GIS perfectly. In other words, each of these images is perfectly aligned in real-world space. Here is another close-up of one of the squares. now, as useful as the two dimensional orthophotos are for making measured drawings, the 3D models bring the site to life, but also offer new perspectives of the excavation that would otherwise be impossible to represent. We can move around the 4 x 4 meter square and inspect the architecture from any angle. This is especially important in archaeology, where we really do have just one chance to get it right, since the square has been fully excavated and no longer exists.
This imaging technique is also useful for creating imagined places. It was used to develop the Star Wars game battlefront, where the team would go out to locations, such as this forest and take hundreds or thousands of photos to produce highly realistic backdrops and 3D models to populate the game. Let’s take a look.
Why 3D Imaging?
Now granted, not everyone is interested in producing 3D blowup games, so what are the motivations for 3D imaging? Once you realize how easy the technique is, it becomes apparent that taking more pictures, and especially photographing 3D imaging sequences becomes a way to produce archival images and knowledge in digital format that often is irreplaceable we will take a look at some more examples in a moment these techniques will generate measurable models with exceptional precision and accuracy. This, combined with some care around collecting information about color, texture and other conditions produces a remarkable record the sequences are not difficult to replicate, so often photogrammetry is used for the documentation of objects, sites and sacred places that are often at risk, or for recurrent condition assessments, such as documenting rock art that may be damaged over time due to weather, vandalism, or other factors 3D imaging is particularly effective for education and research and sharing and,it’s also a lot of fun.
Strategies for Using Digital Photogrammetry in a Project
Now, we’re going to take a slightly deeper dive into understanding what is necessary for conducting successful photogrammetry projects. I would like to thank the team at Sparc at the University of Arkansas for providing some of the content for this tutorial that we are about to explore. What is photogrammetry? The “art, science and technology of obtaining reliable information about physical objects in the environment through the process of recording, measuring and interpreting photographic images and patterns of electromagnetic radiant imagery and other phenomena.” – (American Society for Photogrammetry and Remote Sensing 1980)
Photogrammetry predates photography. The principles go all the way back to daVinci. We have provided a timeline of the historical events
that have led to the modern advances in photogrammetry. This is available as a presentation, as well as an interactive timeline in the iBook version of this tutorial (coming in part II). We would highly recommend checking it out and taking five minutes to immerse yourself in the history of this fascinating scientific breakthrough.
A couple of brief highlights:
In the early days, when we talk about photogrammetry, we’re really talking about aerial imaging. Generally this was done by creating stereo pairs, two images with 80% overlap that, when combined, would produce a 3D effect. Because many of the features in each image are represented in both, those differences can be measured, producing orthographic images. As we zoom forward to the 1990s, computer vision afforded new capabilities for computer-based systems, but the systems were extraordinarily expensive. Generally metric cameras, that is cameras that are specifically calibrated for the purposes of aerial imaging were deployed, but again, this required very complex processes, expensive equipment and a highly trained personnel with basically no room for human error. You either got it right, or it didn’t work. And then… An algorithm was invented that dramatically improved the processing of images and basically broke the mold for how images were and are generated. Instead of following very strict rules of position, this technique of feature mapping depends on movement. In fact, it is based on the principles of structure from motion. So how does this work? Well, it is similar to how we perceive the world. The more we move with our two eyes, the better perception we have of the dimensionality of the subject. In other words, multiple vantage points from multiple cameras, in this case, renders a far better result than less images from particular, static points of view. This makes it possible for humans to be highly effective generators of precise 3D imaging for the first time. And, it comes naturally because it’s similar to how vision works for us in our everyday lives. And so, the SIFT algorithm was the breakthrough, and everything has changed dramatically ever since. Now, automated close range photogrammetry is totally commonplace in documentation projects. Now is the time to learn this remarkable technique.
What is Necessary
Let’s talk about the essentials. First off, you will take lots of photos. Lots and lots of photos from various angles, depending on what you are documenting. Essential to this is to have continuous photographic overlap between shots of close to 80%. That is, photos are taken in a series, where each photo overlaps the previous one, and the goal is to make sure that all of the photos in the set overlap other photos in the set. This is where the magic happens. The software, guided by information derived from within the photos, the technical metadata that is captured automatically by the camera, plus some guidance by you, the human operator, processes the overlapping photos taken from different locations and automatically matches features that it finds between photos. The good news is that if you follow the rules, this entire process is almost completely automated. However, to derive an excellent quality 3D result, there is work to be done, which includes eliminating poorly, or incorrect matches between photos and other steps. But let’s not worry about that right now.
So, contiguous photos and lots of movement. In this example, we are dealing with a very complex, multidimensional objects. This is not a flat wall. It’s going to require a lot of photos with a lot of overlap, taken from various angles to make sure that nothing is missed. It’s essential that you move… Many people who begin learning photogrammetry get confused because the second most popular technique for overlapping photographs is producing panoramas. With panoramas, you get a better result by not moving, by staying in place and simply moving the camera around in a circle around the nodal point, as it is called a single point in space, like the top of the tripod. However, in order for the software to produce depth information, you need to move. If you close one eye and just move your head, you will get the picture. Depth perception requires movement, and it is no different here. Lighting is also really important. The better the lighting, the better the result. Minimizing shadows and generally having even, diffuse light makes a huge difference in the resulting imagery. As you can see in these photos, sometimes all it takes is a large shade and a person to move it, because you will be moving. You can take the shadow with you. I personally also like to use strobe or flash with a diffuser as you see below. But this is also a topic for another time.
Mark Letteney helps set the stage for a large area photogrammetry session at the Roman camp of Legio VI Ferrata in Israel. Courtesy of the Jezreel Valley Archaeology Project.
Now, we’re going to talk about the essentials for the equipment. You really can get started with a modest camera, but you are not going to get very good results if you try to do this with a tablet such as an iPad or a smartphone, for reasons I will get into in a moment. You need:
- a digital SLR, or mirrorless camera. In other words, a decent point-and-shoot, or a camera that can accommodate multiple lenses. Preferably a camera that can shoot in camera RAW format and with a decent sensor. We will cover all of this in just a minute.
- And, a color checker, some form of known color chart that can be stuck somewhere in the scene.
- The same goes for an object of known length… This is super important, some form of scale bar that can be used for deriving the actual size of the subject you are recording.
When evaluating a camera, these are the things to look for. It is almost impossible to buy camera these days that has less than eight megapixels. I would target 12 megapixels or above. The sensor size is super important, and I will cover that a bit more in a moment. It is ideal that you can set the focus of the lens manually and not use autofocus. This is another reason why smartphones are a hassle, because it’s very difficult or impossible to manage focus. Whenever possible, you want to shoot in camera RAW format. For the purposes of this tutorial, just trust me… It’s truly the difference between an original painting and a xerox copy in terms of the actual image quality. Hard drive space is cheap in comparison to a lost opportunity to produce 3D imaging of the subject that might be lost or damaged.
Other accessories to consider are our tripod or monopod, remote cable release or wireless shutter, and using a flash or strobe unit when appropriate.
Sadly, inexpensive cameras and almost all smartphones and iPads and other tablets use something called a rolling shutter. Basically, the way that the pictures taken by the camera renders an image that the photogrammetric software doesn’t know how to handle very well, and this often leads to poor model quality or complete failure. This isn’t to completely discourage you from trying, and there is more and more software coming out that is specifically designed for smartphones. However, at the time of this video in the early part of 2016, we would discourage using a smartphone or tablet as your primary documentation method and use a dSLR or mirrorless camera. The results will be infinitely better.
Now, deep within each picture taken with most devices is a whole lot of technical information that’s pretty spectacular. Timestamp, the model of the camera, lens information coordinates, etc. and all of this information helps the software produce better models faster. It’s really important to use processing software, such as Adobe Lightroom, that does not erase this information.
A couple more tips. If your camera has digital zoom, don’t use it. In fact, just don’t ever use it, ever. All digital zoom is doing is cropping your image beyond the resolution possible optically. This introduces a whole lot of noise and other artifacts into the picture that generally make them look terrible. Just use the maximum zoom available of your lens and crop later. For the purposes of photogrammetry, you are generally going to be using a wider lens, so this really doesn’t apply. If at all possible, use what is called a prime lens, a lens with a particular focal length, such as 50 mm or 28 mm. If all you have is a zoom lens, you’re going to set a particular focal length, and lock it down. I like to use painters tape, that blue tape that is easy to remove right on the barrel of the lens. The same goes for focus. Once you have said it, don’t change it. And, if your camera has image stabilization, turn it off because it introduces a little bit of movement in the picture that causes the software fits. Megapixels aren’t everything and this is really confusing for everybody.
The actual size of the sensor also matters a great deal. A typical smartphone has a very small sensor, which makes sense since it’s a very small device and is not a whole lot of room. Ideally, you want to have a camera with at least an APS – C size sensor. These are found in most SLR type cameras. However, many mirrorless cameras also have decent size sensors. I personally like to have a full frame sensor, but the reality is that full frame cameras start at around $1500 and up. You really can get a fantastic result with something like a Canon rebel or other cheaper digital SLRs. Will talk a bit more about these at the end of this tutorial.
The final model resolution, that is, the quality and precision of the model, depends on the pixel size, focal length in working distance combination. If you think about this for a moment, it makes sense. The sensor defines the pixel size, if you use a very wide-angle lens, the amount of resolution will be less, and the further you are away from the subject the less precise everything is going to be. Generally, there is no need to get obsessive about this. The more you practice, the more you will come to an understanding of the right combination of these factors. It is out of the scope of this tutorial, but there are many times that a smaller sensor will actually produce a better result, so stay tuned.
Data Processing, the Quick Tour
Without going too deeply into this, let’s take a look at the data processing. You have taken your photos, you have processed them through your imaging software, our preference being Adobe Lightroom or its nondestructive equivalent, and you have produced a set of images and brought them in the photogrammetry processing software. This is a screenshot from AgiSoft Photoscan, the only software we would recommend using if you are new to this game. We’ll talk a bit more about software in a moment.
This is the basic processing pipeline.
- You take the photos, bring them into the photogrammetry software.
- The software aligns the photos and generates a point club, a collection of points in 3D space that has magically been derived from the images.
- These are further processed into a dense point cloud and from there, a mesh is created.
- Then, the texture and color information is added, along with scaling the model or using geo-references from a GPS or other device.
- And finally, contextualizing and annotating the model so that it is useful for others.
This may seem like a lot of steps, but all of these are done within the software package, so don’t panic. Let’s take a quick step back. I mentioned earlier that it is important to have a color checker somewhere in the scene. If you are interested in accurate color of your subject, and you should be, this is an essential step. Photo scan will actually try to fix the color for you if you want it to try, but I personally prefer to have complete control this process so once again, Adobe Lightroom to the rescue. Also, if you photograph with color balance in mind, the whole process is simply so much easier to deal with. We will cover color management in another tutorial.
Let’s demystify everything. Here we see all of the cameras, the blue boxes represent each camera, from 186 photos that have been aligned inside of the software, notice the tremendous overlap between photos. The next step is to generate the sparse point cloud. Believe it or not, this is as easy as clicking a button inside of the software. What you are seeing is not a photograph, but actually the underlying 3D data as a cloud of points. now, additional processing will bring the sparse point cloud up to a dense set of points, you can really go crazy with this and generate millions of points or more, but generally, 1 million points or less is fine. next, the software will generate a triangular mesh by joining the points together. In this model, there are now over 12 million faces connected by over 6 million vertices. the final processing step is to add in the texture data, and will cover that result in another tutorial as well.
Congratulations, those are the basics. For review, to really do this right, you will need some modest gear. A color checker. Small versions of the colorchecker shown here or as little as $10 on amazon.com. You will need an object of known length, something that you can use to scale the model. Shown here are very cool scale bars produced by Cultural Heritage Imaging which include these special targets that the software can read, and this makes everything a whole lot easier. You can create your own. You can also simply use other types of scale bars or any object that is easy to see in the photo and that you can measure. Make sure that you shoot in camera Raw Whenever Possible. This is worthy of a complete additional tutorial. To ring in hope, I will mention that this camera, the Sony NEX six, has all of the features I mentioned. 12 megapixels, it shoots camera raw, it contains an APS – C sensor, choose manual focus, it has interchangeable lenses and can accommodate a fixed lens. And it’s available on eBay for less than $300. I own three of them. In terms of shooting, you’re going to want to lock your focus, lock your zoom, shoot with a fixed aperture, preferably F/8 or smaller. you want to lock the speed to something lower than iso–400 and whenever possible, you are also going to want to lock the exposure. This last one is tricky, and in the real world we typically let the camera choose the exposure but we lock everything else down. And as I mentioned, iPads and iPhones and other smart devices are not really up to the task, but they are fantastic for documentation. We use them to collect all of the other information that you’re really going to want to have, such as the time of day, weather conditions, the purposes of the imaging, any special circumstances, etc. Finally, just to touch on the preview of an upcoming tutorial on born-archival imaging, I want to leave you with a sense of excitement and encouragement. We have successfully taught many people around the world how to do this technique. You are not alone, there is tons of help out there and everything is getting easier and better all the time. We really feel that this is the next revolution in photography, but more generally in culture heritage documentation. Once you try it, it is unlikely you won’t love it. This presentation is part of a series of training materials, so I encourage you to check out the whole series and get in touch with any questions you may have. We would love to hear from you.
Special thanks to:
Adam Prins, Jezreel Valley Regional Project
Adam Barnes, Katie Simon, Adam Wiesel, Rachel Opitz, SPARC @ Center for Advanced Spatial Technologies
A Brief History of Aerial Photogrammetry, aer0metrex_admin
Due to the heavy rainfall this winter, California State and National Parks and Heritage Sites are under threat of serious structural damage, mud slides and land erosion. The Ghost Town of Bodie is among these.read more
Freelancer Developers can be an affordable option for small teams or projects with a tight budget. CoDA tested out Upwork, an online hiring platform, last year and worked with a few freelancers, some better than others! Here are 5 things we learned and how you can get the most for your project on a budget of any size.read more
Three organizations team up to explore possibilities for international exchange and increase impact of world heritage and archaeology. Great news! These three organizations have started a collaboration through a Memorandum of Understanding to establish a cooperative...read more
For the ancient Polynesians, folklore was shared by word of mouth, today the most popular platform for telling stories are movies. Moana, Walt Disney Studios’ latest film, tells the story of a young girl raised to become chief of her people who comes to discover she...read more
The Hawaiian people built a rock enclosure in a shallow depression in a volcanic ridge in the mountains above Kapolei, in an area now known as Pālehua.read more
This course is a cross-disciplinary exploration of Cultural Heritage on a global and local scale through discussion, debate, in-class activities and team-based research projects.read more
Six Fundraising Lessons from the world most successful fundraisers at the 2016 “Nonprofit Fundraising Masters” event in San Franciscoread more
Looking back on the process, Kodak PCD conversion almost seems easy. If you are at the beginning of your zip recuperation journey, these steps may not be obvious, but they are very doable. The only solution available which will keep images high-resolution, convert and maintain original metadata, and save you a LOT of working hours is PCD Magic!read more
At the Lowest Place on Earth, on the South-East shores of the Dead Sea, nestled in the Ghor es-Safi and a short turn off the Dead Sea highway, you’ll find a little museum with a big mission and big stories to tell.read more
The Lifeways of Ancient Africans offers an overview of the archaeological heritage of Africa, from the emergence of modern humans to the beginnings of European colonization.read more