You’ve heard it in every boardroom and blog: “We’re adding AI to our platform!”
...Cool story. But when AI meets complexity without a plan, what you get isn’t magic - it’s spaghetti (and not a good kind...)
This is where expert-in-the-loop (EITL) systems shine - it's the much-needed middleware between raw automation and real-world decision-making, especially in the realm of Platform Engineering. And when done right, EITL doesn’t just make life better for developers—it sets the stage for a smarter, more adaptive organization overall.
What Is “Expert-in-the-Loop”?
At its core, expert-in-the-loop is a simple concept: machines do the heavy lifting, but humans keep their hands on the wheel. It’s like using cruise control—you’re still driving, just with a bit less footwork.
In Platform Engineering, EITL can mean:
- AI auto-generating Terraform templates… but an engineer approves and modifies them.
- LLMs summarizing incident root causes… but an SRE signs off before postmortem docs go live.
- GenAI chatbots answering developer questions… but escalating unclear queries to human experts.
Rather than chasing fully autonomous ops (we’re not there yet, folks), EITL acknowledges that context and judgment are still human superpowers.
From DevEx to… OrgEx?
Now, you might be asking yourself: “Wait, what’s OrgEx?”
Glad you asked. It’s a term I like to use when Developer Experience (DevEx) isn’t just about better CI pipelines or cleaner onboarding docs—but becomes something bigger.
OrgEx is what happens when improving DevEx starts to ripple outward: smoother cross-team collaboration, fewer silos, and institutional knowledge that actually sticks. When you start putting experts in the loop with AI, you’re not just fixing developer pain points—you’re untangling the messiness of how an entire organization functions.
Think of it as DevEx, but scaled to the organisation level.
The Current Naivety: AI Isn’t a Shortcut
Here’s the kicker: a lot of companies are throwing GenAI at massive, tangled problems and expecting miracles. Spoiler: it doesn’t work like that.
The hard truth? Complex organizational problems rarely have monolithic solutions. They need to be broken down—refined, scoped, debugged like any good codebase. That’s where the real engineering work lies. AI can support the solution, not be the solution.
Expert-in-the-loop reminds us to ask: What does the machine do well? What should a human decide?
Skipping this step? That’s how you end up with AI generating Terraform files that accidentally nuke your staging (or maybe even production, depends how close to the edge you like to walk...) environment.
What’s Next: Rolling Up Sleeves with GenAI, DevEx & OrgEx
We’ve talked theory—now let’s get our hands a little dirty.
Recently, I joined Vandebron, where we’re in the middle of introducing Backstage and a suite of modern technologies aimed at leveling up both Developer Experience and broader organizational flow. It’s a rare opportunity: greenfield meets real-world complexity. And with that comes a chance to explore how GenAI can fit into the messy, wonderful world of Platform Engineering—not as a silver bullet, but as a genuinely helpful tool.
Over the coming posts, I’ll dive into a few hands-on areas I’ll be experimenting with as we build out these systems:
🛠️ Terraform Feedback Loops with GenAI
What if an LLM could scan your terraform plan and call out “Hey, deleting that RDS instance might be a bad idea”? We’ll explore how to plug GenAI into infrastructure workflows to flag risky changes and suggest safe patterns—before disaster strikes.
💬 Backstage Chatbot for Onboarding and Org Clarity
Getting new devs up to speed shouldn’t feel like decoding ancient scrolls. I’ll be prototyping a Backstage plugin that lets folks ask, “Who owns this service?” or “How do I get deploy access?”—and get real answers from an AI that’s been trained on your org’s context.
🧯 Postmortem Summaries from Logs, Metrics & Traces
Writing incident reports is nobody’s idea of fun. Let’s see what happens when GenAI takes in your telemetry and drafts the first cut of your postmortem—timeline, impact, root cause—so humans can focus on insights, not formatting.
📚 Living Knowledge Management (That Doesn’t Rot)
Documentation usually dies a slow, painful death. I’ll be testing ways to use GenAI to keep internal knowledge fresh—automated Q&A synthesis, wiki generation, and feedback loops that actually improve with time.
This is going to get fun!
This isn’t just a tech experiment—it’s a journey to figure out how AI can actually help developers, not distract them. Whether you’re knee-deep in Platform Engineering or just trying to untangle org spaghetti, stay tuned.