Automating Multi-Cloud Assessments with Python
When you're managing infrastructure across four cloud providers — GCP, Azure, AWS, and OCI — manual inventory is not an option. At Matilda Cloud, our customers needed a clear picture of their entire cloud estate before making migration or optimization decisions. The old approach involved weeks of spreadsheet wrangling and manual portal clicks. I set out to automate the entire process.
The Problem
Enterprise customers rarely live in a single cloud. A typical engagement might involve 200+ Azure subscriptions, dozens of AWS accounts, several GCP projects, and a handful of OCI tenancies. Before you can recommend a migration path or identify cost-saving opportunities, you need to know what exists: every VM, database, storage bucket, network, Kubernetes cluster, and their configurations.
Manually cataloging this across four providers with different APIs, authentication models, and resource hierarchies was taking our team 2–3 weeks per customer. That's unacceptable when you're scaling the platform to handle multiple customers simultaneously.
The Architecture
I built the discovery platform in Python, with a modular provider layer so each cloud's SDK calls are isolated behind a common interface. The core components are:
- ▹Provider Adapters — Thin wrappers around each cloud SDK (google-cloud-python, azure-sdk, boto3, oci-python-sdk) that authenticate, paginate, and normalize resource data into a common schema.
- ▹Discovery Engine — Orchestrates parallel discovery across subscriptions/projects/accounts, handling rate limits and retries.
- ▹Data Layer — MongoDB for flexible document storage of discovered resources, Elasticsearch for fast querying and reporting.
- ▹REST API — FastAPI endpoints that expose discovered inventory, cost data, and optimization recommendations to the frontend.
Handling the Complexity
Each cloud provider has its own quirks. Azure resources are organized by subscription and resource group. AWS uses accounts and regions. GCP has projects and zones. OCI has compartments. The normalization layer maps all of these into a unified hierarchy so downstream consumers don't need to think about provider-specific structure.
Rate limiting was another challenge. GCP's APIs are particularly aggressive with quota enforcement. I implemented exponential backoff with jitter and a token-bucket rate limiter that respects each API's published limits. For large estates, the discovery runs across multiple threads with a shared rate limiter per provider.
FinOps Integration
Beyond just discovering resources, the platform pulls billing data through each provider's Cost Management APIs — GCP Cloud Billing, Azure Cost Management, AWS Cost Explorer, and OCI Cost Analysis. This lets us correlate resource inventory with actual spend, identify idle resources, and flag right-sizing opportunities automatically.
The billing pipeline aggregates cost data at the resource level, tags it with the discovered metadata (owner, environment, application), and produces daily cost reports. This alone saved one customer $340K/year by identifying 47 oversized VMs and 12 unused databases.
Results
What used to take 2–3 weeks of manual work now runs in under 30 minutes for a typical enterprise estate. The platform has processed over 50 customer environments, discovering millions of resources across all four cloud providers. It's become the foundation for every assessment, migration, and optimization engagement we do at Matilda Cloud.
The key lesson: invest in the abstraction layer early. The time spent building clean provider adapters paid for itself many times over when we needed to add new resource types or support new provider APIs.