No new features were shipping. No customers were onboarding. EventBridge fired maybe once a week—sometimes less—triggering a Lambda that published messages into three SQS queues. Two services would wake up, do their work, and go quiet. A few minutes later, a third service would pick up its delayed message and finish the sequence. CloudWatch graphs were flat in the reassuring way that usually means nothing is wrong. Operationally and from a business point of view, nothing meaningful was happening. ...
Catching bad geometries at ingestion is only half the problem. The other half is knowing when your transformations corrupt them. I wrote previously about silent errors in geospatial pipelines - geometries that pass validation but produce wrong results. Those errors typically happen at the boundary of your system, when you’re loading data from external sources. You can catch most of them with better validation: check winding order, enforce CRS consistency, flag suspicious areas. ...
The hardest bugs in production geospatial systems are not the ones that crash your pipeline. They’re the ones that let it keep running. I spent the last few years building systems that process millions of agricultural land parcels. The work involves dozens of data sources, multiple coordinate systems, and complex validation rules. In that time, I’ve learned that the most expensive problems are the ones that pass all your checks but still produce wrong results. ...
Welcome to my technical blog! I’m Gokul Ganesan, a geospatial data engineer with experience building systems that serve billions of requests and process petabytes of spatial data. What You’ll Find Here This blog focuses on practical insights from building production geospatial systems at companies like Mapbox, EagleView, and Sagri. I’ll be sharing: Technical Deep-Dives: Detailed explorations of geospatial algorithms, data structures, and optimization techniques Architecture & Performance: How to build scalable systems that handle massive spatial datasets Geospatial Engineering: Best practices for working with spatial data, from ETL to visualization Sustainability Tech: Using technology to solve environmental challenges Industry Insights: Lessons learned from building production systems My Approach Every post will be grounded in real-world experience. When I write about optimizing a spatial index or designing a distributed processing pipeline, it’s based on systems I’ve actually built and the challenges I’ve faced. ...