Every Photo Has a Cost

Why drone inspection planning is a data management problem, not a navigation problem

In my previous post, I talked about how I stopped thinking of mission planning as simply making a drone fly from one waypoint to another. The real challenge was making inspections repeatable.

That realization changed the way I approached the project, but it also exposed another problem that I hadn't anticipated.

The hard part didn't end when the drone landed.

In many ways, it was just beginning.

When we started using our planner in real inspections, we quickly ran into a problem that had very little to do with navigation.

We were using the DJI Mavic 3T, and every capture generated two images: one in the visible spectrum and another in infrared. Both were equally important for the inspection, but they also meant that data accumulated much faster than I had expected.

A route with a few hundred waypoints could easily produce hundreds of images.

Now multiply that by multiple substations, recurring inspections, and thousands of assets.

The question was no longer:

"Where should the drone fly?"

It became:

"How do we reliably manage and correlate all of this data?"

Eventually, I realized the problem had two very different parts.

The first was storage.

The second was correlation.

Storage turned out to be the easy one.

Not because it wasn't important, but because we had people on the team who already knew how to build that kind of system.

Antônio Gustavo was leading much of that work. He built the kind of backend infrastructure nobody thought much about—which, in retrospect, was exactly the point. It simply worked.

João Pedro joined the team as an intern and quickly became an important part of that effort. It was rewarding to watch him grow into someone who took ownership of production services and helped evolve the platform alongside the rest of us.

Because of them, storage never became the bottleneck.

We all had a shared understanding of how files should move through the system and how an event-driven pipeline could process them.

Correlation was different.

We all knew how to build reliable storage systems.

Nobody on the team had solved the correlation problem before.

Our storage pipeline followed a fairly traditional event-driven architecture.

Once media was uploaded, we committed the upload and emitted an event for each file. From there, a choreography of services persisted the media, extracted metadata, identified its type (RGB, infrared, zoom, panorama, etc.), stored its reference in the database, associated it with the current inspection, and once correlation completed, the media was reorganized into a predictable S3 hierarchy.

A simplified layout looked something like:

The architecture matched the problem well.

Each service had a single responsibility, and queues allowed us to process thousands of images while scaling each stage independently.

At first glance, the pipeline felt solid.

It was reliable, scalable, and relatively straightforward to maintain.

Then reality started creeping in.

As inspections grew and temporary failures became more common, we occasionally noticed media disappearing from the pipeline.

The uploads had succeeded.

RabbitMQ hadn't lost any messages.

The consumers had received them.

The problem was much more subtle.

One subtle behavior in the underlying messaging stack meant messages could be acknowledged before the processing we cared about had actually completed. If processing failed afterwards, the broker had already considered the work complete.

From RabbitMQ's perspective, everything had worked correctly.

From ours, an inspection image had silently vanished.

We eventually redesigned that part of the pipeline to acknowledge messages only after processing completed successfully.

It sounds like a small implementation detail.

It wasn't.

That change fundamentally altered how I think about building reliable event-driven systems.

The architecture that emerged from that redesign is something I'd like to explore in a future post.

Even after solving that problem, storage still wasn't the hardest part.

The real engineering challenge wasn't storing the files.

It was understanding what every file actually represented.

At first, we tried solving the problem during data collection.

Could we use filename suffixes?

Different folders?

Additional metadata?

Special naming conventions?

Every solution looked reasonable until we measured its impact.

Each additional action added friction to the recording process.

And in drone inspections, every unnecessary second matters.

More time spent interacting with the application means less battery available for collecting useful data.

None of those approaches felt reliable enough.

Our next attempt was to map capture points directly to inspection assets.

After completing the flight, the pilot would associate each capture point with its corresponding asset.

It worked surprisingly well during early tests.

Until the routes started changing.

A new waypoint meant updating the mapping.

Removing a waypoint meant updating the mapping again.

And once we were dealing with thousands of inspection points, maintaining those relationships became slow, repetitive, and increasingly error-prone.

It solved the immediate problem.

It didn't solve the scaling problem.

The turning point came after a conversation with Grubert.

We were going over yet another attempt to improve correlation when he pointed out something that, in hindsight, sounded obvious.

We already knew where the drone was.

We already knew exactly how the camera was oriented.

We already had an orthophoto of the substation.

And inspection assets rarely changed their physical location.

None of those pieces were particularly useful in isolation.

But once we started looking at them as parts of the same system instead of independent pieces of information, the solution began to emerge.

Instead of trying to associate images with assets during every inspection, we could create the inspection points once, directly on the orthophoto, and treat them as fixed references.

That alone didn't solve the problem.

But it completely changed how we approached it.

At first glance, computer vision seemed like the obvious solution.

Train a model to recognize every inspection asset and let it decide what each image contained.

The more we discussed it, the less attractive that approach became.

We would need to collect and annotate thousands of images, train models, validate them, and then run inference over every inspection. All of that required infrastructure, compute resources, and processing time that simply didn't fit the operational window we had.

More importantly, it felt like we were solving a much harder problem than necessary.

We weren't trying to understand the contents of an image.

We were trying to answer a much simpler question.

What was the camera actually looking at?

Those are very different problems.

Once we reframed the question, geometry became a much more practical solution than computer vision.

That's when virtual cameras entered the picture.

Since we knew where the drone was and exactly how the camera was oriented when every image was captured, we could recreate a virtual camera inside the mapped environment.

Now the question became much simpler.

Given this camera, which asset was it actually looking at?

My first instinct was to pick whichever asset was closest to the center of the image.

It sounded almost too simple.

Unfortunately, reality got in the way again.

Because we were working with a 2D representation of the environment, depth information simply didn't exist.

An asset might appear close to the center while actually being behind another structure.

Without the third dimension, "closest to the center" wasn't enough.

We needed another constraint.

What ended up working surprisingly well was combining two simple geometric rules.

The selected asset had to be both:

• close to the center of the image; and
• close to the camera's position.

Neither rule was sufficient on its own.

Together, they dramatically reduced incorrect correlations.

No computer vision model.

No expensive inference pipeline.

Just careful use of geometry and the information we already had.

One unexpected advantage of this approach appeared later.

Since inspection points became fixed references in the environment instead of being tied to specific waypoints, routes could evolve without breaking the correlation pipeline.

We could add, remove, or adjust capture points while still maintaining a reliable relationship between historical images and inspection assets.

That flexibility turned out to be just as valuable as the correlation itself.

In practice, this entire acquisition and processing pipeline reduced the overall inspection process time by roughly 95%.

What I find interesting is that the biggest improvement didn't come from a sophisticated algorithm.

It came from realizing we were trying to solve the wrong problem.

For a long time, we focused on making image correlation more reliable.

Eventually, we realized correlation itself wasn't the problem.

The real problem was representing the inspection in a way that made correlation almost trivial.

Looking back, that's probably the lesson that stayed with me.

Good systems rarely emerge from optimizing one part of the system in isolation.

The planner, the processing pipeline, the storage strategy, and the inspection workflow all depended on each other.

The solution only emerged once everyone—from pilots and inspectors to software engineers—agreed on what the system was actually trying to achieve.

When I started working on the planner, I thought the question was:

"How do we make the drone fly this route?"

By the end of the project, I realized the real question was:

"How do we make sure every image still has meaning six months from now?"

Those sound like similar questions.

They aren't.