Cloud graph and the cloud-to-code agent: discovery and codification

TL;DR

Real cloud estates are full of resources nobody wrote as code.

I designed the loop that finds them and turns them into managed infrastructure: a legible cloud graph to see the estate, a discovery explorer to import live resources, and a codification agent that turns discovered resources into Terraform and opens a PR.

The problem

Two hard things at once.

First, a real topology is a hairball of hundreds of nodes that no visualization made legible.

Second, huge amounts of infrastructure exist in the cloud but not in code (unmanaged, undocumented, drifting), and hand-writing IaC to adopt it is slow and error-prone.

The insight

A graph is a tool for answering a question, so it should start scoped to the question, not render the universe.

And adoption is a pipeline (discover, decide, codify, review) with a human gate at the review step, because you are about to take ownership of real resources.

Early on I prototyped a pure force-directed layout and dropped it: it produced a different unstable hairball on every render, so I moved to a force graph with a dagre pre-layout that gives stable, readable positions.

The solution

• Cloud graph: a D3 force-directed graph with dagre pre-layout (stable, not random), a shared node-card primitive, spotlight and fade to focus the subgraph relevant to the question, and sanitized labels so raw graph terminology never leaks.

terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 planstate + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { }vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9

• Discovery explorer: browse a live cloud account's resources, multi-select, and import them into an environment, with a unified scope selector across collections, repositories, and branches.

terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 planstate + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { }vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9

• Codification agent (cloud-to-code): the agent runs the pattern end to end, from discovering a resource to comparing its live state against code, generating the Terraform change, opening a PR, and deploying. A resource-decision block handles the unmanaged or drift call (import, reconcile, or mark expected), and a code-indexing block makes ingestion legible while it runs.

terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 planstate + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { }vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 planstate + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9 vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { }vpc deny state + ~ terraform module ingest lock 0 1 apply drift graph { } 9fa3 plan iam helm => b4f aws_s3 allow k8s == 2f9The whole loop in motion: from the estate graph, through discovery and import, to codified resources with zero drift

Impact

Turned shadow infrastructure everywhere into a guided, reviewable path from live cloud to owned code.

Reflection

The visualization win was editing, deciding what to leave out by default.

The codification win was framing adoption as a reviewable pipeline, not a magic button, so people trusted the agent to write their infrastructure.