Drones Are Turning Photogrammetry Into Big Business

*Article originally posted by builtin

This past January, at a clay mine near Golden, Colorado, Alena Iskanderova made a startling discovery: The tracks of an ancient relative of the crocodile — once preserved for some 100 million years — had been largely erased by erosion.

In the 11 years since paleontologist Martin Lockley, an associate curator at the University of Colorado Museum of Natural History and professor emeritus at the university, first documented the tracksite, the fossilized footprints left by the animal had lost much of their depth — from roughly 7 to 12 millimeters down to 3 to 4 millimeters, Iskanderova said.

That the effects of the elements could visibly diminish the tracks in such a short time points not only to their fragility in the face of climate change and other anthropogenic threats, Iskanderova said, but the importance of photogrammetry as a means of preserving the geologic record.

WHAT IS PHOTOGRAMMETRY?

Photogrammetry is the science of reconstructing objects and environments in the physical world through photographs. The technique involves stitching together large collections of overlapping photographs to create topographical maps, meshes, and lifelike 2D and 3D digital models. Software tools help create these digital assets using pixel data from aerial photographs taken by drones or close-range photographs with handheld cameras. From surveying construction sites and flood zones to exploring fossil sites and assessing crop health, the technique has a variety of applications.

“Sometimes tracks are the only presence of animals in any paleoenvironment. So [photogrammetry] is very important for us to know what kind of animals were there,” she said. “The tracks also show us their behavior. Sometimes we can tell, for example, that there was a group of dinosaurs migrating from one point to another.”

Iskanderova is a close-range photogrammetrist with a specialization in paleontology. She uses a Canon 5D Mark II camera with a 24-mm lens to do much of her work, which has included documentation of ornithischian (“bird-hipped”) dinosaur tracks, small invertebrate burrows and the first reported Mesozoic track of a small heron-like bird called Ignotornis mcconnelli. Most of her work occurs in the South Platte formation — a sandstone-rich rock bed in the foothills of Colorado’s Front Range.

“Sometimes tracks are the only presence of animals in any paleoenvironment. So [photogrammetry] is very important for us to know what kind of animals were there.”

After snapping hundreds of overlapping pictures, Iskanderova uses Agisoft Metashape Pro 1.7 to patch them together into a single 3D image. By aligning pixels in the photos, the software renders something called a point cloud — a three-dimensional constellation of colored dots that reveal the contours of a surface. These points are then layered with a textured mesh to create lifelike visualizations, including depth maps showing the geolocated contours of a surface.

“This is why photogrammetry should be taken as a best practice for fossils and tracks studies,” Iskanderova wrote. “[Many scholarly] papers give measurements as numbers but don’t document how the measurements are made in connection to the start and end points. Each scientist, or a fieldwork assistant, will take the measurements differently. With photogrammetry, you can record not only the length, width and depth [of tracks] but also the start and end points.”

MORE ON 3D VISUALIZATIONSDigital Twins: What They Are and How They’re Shaping the Future

Ground Control to Major Robot

Photogrammetry is nothing new. The centuries-old method of reconstructing measurements is rooted in principles of perspective and projective geometry practiced by artists, such as Leonardo da Vinci, and formalized into a science by German mathematicians Rudolf Sturm and Guido Hauck in the late 19th century. Yet the field is rapidly evolving through innovations in software and aerial photography.

Today, photogrammetry is used in commercial applications as diverse as public safety, construction, civil engineering, automotive manufacturing, agriculture and military reconnaissance. And a growing number of use cases has been a boon for the software companies that provide 3D modeling and post-production tools.

Analyses from Data Bridge Market Research predict the photogrammetry software market will see a compound annual growth rate of more than 13 percent between 2021 to 2028, with photogrammetry software expected to reach a market value of $2.56 billion by 2028.

“I think the big revolution has been with drones,” Tristan Randall, a strategic project executive at the architectural software company Autodesk, said. “In the context of a construction project, for example, where you want to monitor your site conditions, you can purchase drones that cost a couple thousand dollars. So capturing the photogrammetric data has become much, much easier.”

Photogrammetry does not require highly sophisticated cameras, Randall told me. It can be performed using digital single-lens reflex (DSLR) cameras, video reels, satellite photos or even images captured with an iPhone — virtually any digital camera that can store multiple images.

“I think the big revolution has been with drones … capturing the photogrammetric data has become much, much easier.”

But the low-cost availability of drones has opened a once largely terrestrial application to a range of new airborne possibilities — from creating large-scale maps to assess crop health or plan for emergency relief operations to producing lifelike 3D models of buildings, roadways and flood zones.

A photogrammetrist can buy a serviceable drone for as little as $800, said Christopher Kabat, the owner and founder of the drone consultancy ProAerial Media. Once programmed, the drone can capture hundreds of photos of a large-scale real-world environment, like a subdivision or city district, in hours.

Prior to the flight, the pilot selects the flight path and the number of photos the camera will take, based on their desired output resolution. Outfitted, typically, with a 1- to 2-inch diameter camera on a rotating gimbal, the drone passes back and forth over the landscape taking pictures — hundreds of them — for later processing.

“It’s literally taking every image and taking all the pixels in each image and looking for another image with at least three matching pixels,” Kabat said. “And it’s going to do that for every single photo that you have.”

Depending on the goals of a project, teams can use drone-based photogrammetry to create photorealistic orthomosaic maps corrected for the curvature of the Earth, capture valuable volumetric data — like the amount of soil a building team needs to extract to dig a foundation — or generate interior models for virtual home tours on real estate sites like Zillow. Aerial photogrammetry, though, tends to work best for large-scale projects, not fine architectural details, which are often represented with laser scanning.

The Google Earth project to map cities in 3D actually used both technologies, Randall told me, capturing large regions with photogrammetry and applying signature building features with manually scanned data.

“The key thing to remember is that [a point cloud] is a very, very accurate representation of the physical features of a site,” Randall said. “We call it ‘mowing the lawn,’ because you’re basically moving the camera in lines that overlap. And then you use those photos in, say, an engine like Autodesk ReCap, to stitch them together.”

Photogrammetry Has Dozens of Use Cases. But Construction Is Where Most of the Excitement Lies.

Over the past decade, aerial photogrammetry has radically transformed the construction industry. Instead of hiring a survey team to spend weeks photographing a site with GPS-synced tripods, developers can send a drone — like DJI’s Phantom 4 RTK or Autel’s Evo 2 RTK — into the air to capture site conditions in hours, often at a much lower cost, said Ryan Sweeney, a sales manager at the Denver office of the photogrammetry company Pix4D.

Drones are good at capturing high-resolution photographs, in part because they fly so low — a maximum of 400 feet above the ground (or higher, if within range of a structure), compared to at least 500 feet above the ground for a human-piloted plane, as stipulated by Federal Aviation Administration regulations. Drones can also capture a site from multiple vantages and reach places that might otherwise be dangerous for humans to be — like hazardous chemical sites.

At a typical construction site, about 500 images captured by a drone in a 30-minute flight can be processed on a personal desktop computer in roughly two hours, Sweeney said. Flight height, camera quality and the level of photo overlap all affect the quality of the point cloud and final outputs. A 75 percent horizontal and vertical overlap is a good target for a quality data set.

“Implementing photogrammetry gives you the ability to almost have your eyes on location. You can monitor the progress [of a construction site] visually, very simply.”

In addition to knitting the photos together, software modeling tools like Recap, Pix4D, or all-in-one aerial photogrammetry platforms like 3DR and DroneDeploy, align geotagged pixels against cartesian coordinates ground sampled locally or imported from networked GPS data. The reconstructed image files are, thus, correlated one-to-one with their real-world locations — what some refer to as digital twins. These outputs can be represented as 3D building models, topographical maps, depth maps, contour line drawings and 2D orthomosaics.

Because these renderings are accurate to within inches, architects and engineers can use them to update working “as-built” models so they reflect on-the-ground conditions.

If a construction crew moves a planned sidewalk four inches to the west to avoid a root system, the design team doesn’t need to manually update their renderings, Kabat said. They can import updated point cloud data to virtual environments to correct such discrepancies on the fly.

Meanwhile, construction managers can use the 3D models to keep tabs on large-scale development projects, while working remotely.

“Implementing photogrammetry gives you the ability to almost have your eyes on location,” Kabat said. “You can monitor the progress [of a construction site] visually, very simply.”

Laser Scanning vs. Photogrammetry

Laser scanners can produce high-resolution 3D models and maps, often at a higher resolution than what can be achieved using photogrammetry. Yet they tend to be expensive — sometimes tens of thousands of dollars, Randall told me — and they must be moved into position by human operators to “see” their targets.

“You can imagine a construction site 20 miles from the city,” Randall said. “A pilot has to fly all the way from the airport and then go back. Even inside a building, you have to move the scanning instrument all over the site to capture different viewpoints.”

But drone photogrammetry has its limitations too. Most U.S. airports, Kabat said, are surrounded by LAANC (low altitude authorization and notification capability) grids that require formal FAA airspace authorization. A flight can be ground sampled at a given height — say, 137 feet — but fall within a restricted zone that limits the flight ceiling to 100 feet. If not coordinated in advance, that can throw a wrench in a mapping project.

And a recent spate of criminal incidents — drones dropping contraband into prisons and flying within range of airports, leading to shutdowns — have led to more stringent drone flight guidelines, Randall said.

The FAA’s Part 107 guidelines already require all small commercial drone operators to pass a knowledge test and be registered, but a new rule that went into effect in April requires most drones flying in U.S. airspace to be equipped with remote ID. According to the agency’s website, this “helps the FAA, law enforcement and other federal agencies find the control station when a drone appears to be flying in an unsafe manner or where it is not allowed to fly.”

“If you were flying in North Carolina, Illinois, Wisconsin — anywhere there’s much denser vegetation, photogrammetry will never be able to interpret the ground data because it can’t penetrate past the canopy roof of the trees.”

Randall told me it likely implies they have the ability to disable drones that pose a potential threat.

Drones — more specifically, their cameras — also have trouble seeing through foliage. Kabat’s company operates in the desert landscape of southern Nevada and areas of Arizona and Utah, where photogrammetry works well.

“But if you were flying in North Carolina, Illinois, Wisconsin — anywhere there’s much denser vegetation, photogrammetry will never be able to interpret the ground data because it can’t penetrate past the canopy roof of the trees,” he said.

Building edges can also be problematic. Unlike laser scanners, which measure distances as a function of the time it takes light beams to reflect off a target and return to their source, photogrammetry uses pixel matching to approximate distance.

“So depending on what’s in those pixels, you may run into challenges. If you’re shooting from directly above a building, you’re not going to be able to represent that vertical edge with as much accuracy,” Kabat said.

Documenting and Preserving Fragile Environments

But the technology is quickly getting more advanced and adaptable. Newly developed aircrafts scheduled to arrive on the market soon, such as the DJI Mavic Pro 3, are expected to have swappable payloads, Kabat told me, meaning they will let users exchange a standard camera for a zooming camera, thermal imaging camera or LiDAR (light detection and ranging) camera.

The promise of modular camera options is exciting to practitioners like Kabat, but the market for newer technologies will likely take some time to ramp up.

“Most people still don’t even know what photogrammetry is,” Kabat said. “That’s the biggest challenge: just making people aware that you can use photogrammetry to solve problems.”

“It’s not something new,” Iskanderova added. “But in certain fields, like, for example, paleontology, it’s a relatively new and growing field. And, right now, I see many, many students studying photogrammetry and doing projects. Many old-school professors are also interested in photogrammetry.”

“Most people still don’t even know what photogrammetry is. That’s the biggest challenge: just making people aware that you can use photogrammetry to solve problems.”

And it remains an active field for hobbyists. During his off-hours, Kabat traces the history of remote stretches of the American southwest with drones and handheld cameras, capturing artifacts in ghost towns and abandoned mines near Las Vegas, and Native American petroglyphs he finds along the Old Spanish National Historic Trail running from Santa Fe, New Mexico, to Los Angeles.

Recently, he engaged the nonprofit Friends of Pando about the prospect of mapping the Pando, a clonal colony of a single aspen in south-central Utah that looks like a cluster of individual trees, but is connected by a genetically identical root system that spans 106 acres. The threatened tree, among the oldest in the world, has been deemed the heaviest living organism.

“If you were to Google, ‘largest tree,’ it’s still going to be General Sherman, the sequoia tree in California,” Kabat said. “But as far as the largest organism, it’s the Pando. What they’re ultimately looking to do is provide visitors to their website the ability to walk through the aspen forest, virtually, as it changes seasons.”

Though the scale of the project is different, it’s not so far removed from what Iskanderova is doing at a more granular level with dinosaur tracks — reconstructing the fragile outlines of an environment with photogrammetry to document its existence and, hopefully, preserve it for posterity.

“With tracks or any fossils, it’s pretty much detective work,” she said. “You just go in and slowly find more details, making a story behind the remains.”