# Agentic AI is graduating from demos… and it’s already a governance problem

I’m officially tired of the phrase *“agentic AI”* being used as a magic spell.

If your system can plan, call tools, and take actions over time, it’s not “a model feature.” It’s **software with autonomy**. And autonomous software needs:

- a real training stack (not just orchestration glue)
- real evaluation (not just a benchmark score)
- real controls (not just “don’t be evil” in a system prompt)

This week had two releases that feel like the right direction.


## 1) Orchard: open-source… but for training agents, not just wiring them
Microsoft Research’s **Orchard** is positioned as an open framework for *scalable agentic modeling*, with an “environment service” (Orchard Env) designed around reusable sandbox lifecycle primitives. That detail matters: it’s a sign someone is taking “agents run in environments” seriously, rather than treating tool calls like abstract tokens. ([microsoft.com](https://www.microsoft.com/en-us/research/publication/orchard-an-open-source-agentic-modeling-framework/))

Orchard’s headline claims are centered on agent recipes (not just prompting), and it reports strong numbers on tasks like SWE-bench Verified after supervised fine-tuning and RL steps. ([microsoft.com](https://www.microsoft.com/en-us/research/publication/orchard-an-open-source-agentic-modeling-framework/))

My take: **open agent training infrastructure is more important than open agent *demos***. If we can’t reproduce agent improvements without proprietary pipelines, we’re stuck arguing over marketing.


## 2) Agentic CLEAR: evaluation that looks like incident response, not a leaderboard
IBM Research’s **Agentic CLEAR** is an automatic evaluation framework built to generate insights at multiple granularities—**system, trace, and node**—instead of giving you a single pass/fail or score. ([arxiv.org](https://arxiv.org/abs/2605.22608?utm_source=openai))

This is the part I’ve been waiting for:

- **System view**: what is the agent *as a system* doing?
- **Trace view**: what happened step-by-step?
- **Node view**: which component/tool/prompt/module is behaving badly?

Because “my agent got 67% on X” is not the same as “my agent is safe to run against prod.”


## The uncomfortable truth: agents need audits, not applause
The agent conversation is drifting toward **capability theater**:

- A new framework drops.
- A new score is posted.
- Everyone cheers.

But when an agent fails, you need to answer boring questions:

- *What exactly did it do?*
- *Why did it choose that action?*
- *Which component pushed it off the rails?*
- *Can we reproduce it?*

Orchard is about building the agent training pipeline. CLEAR is about making agent behavior inspectable.

Put them together and you get a direction I like:

> **Agents as engineered systems with traceability.**

Not “agents as vibes.”


## Drone parallel: the FAA is literally building policy-as-code for airspace
Here’s the non-obvious connection that made this post click for me.

On **May 6, 2026**, the FAA published a proposed rule to establish a process for certain fixed-site facilities to request unmanned aircraft flight restrictions (UAFRs) for safety/security reasons, with comments due **July 6, 2026**. ([regulations.justia.com](https://regulations.justia.com/regulations/fedreg/2026/05/06/2026-08943.html))

That’s basically a real-world example of: **autonomy requires enforceable boundaries + explicit process**.

Agentic AI needs the same mindset:

- Define where the agent may operate (tools, data, permissions)
- Publish constraints clearly
- Make violations detectable
- Make behavior reviewable after the fact

The FAA isn’t saying “drones are bad.”
It’s saying “drones are real, so governance must be real.”

Same with agents.


## What I want next (and what I’m watching)
If you’re building agents in 2026, here’s the bar I’m starting to expect:

1. **Environments that are first-class** (sandbox lifecycle, reproducibility)
2. **Trace-level evaluation that survives production** (not just dev notebooks)
3. **Failure taxonomies that evolve** (not static rubrics from last quarter)
4. **Controls that are enforceable** (permissions, scopes, tool fencing)

Orchard and Agentic CLEAR don’t solve all of that—but they’re signals we’re shifting from “prompt-craft” toward “systems engineering.”


## Why This Matters For Alshival
Because DevTools is where the agent hype either becomes **developer leverage**… or becomes **developer liability**.

If we get this right, agents become reliable teammates.
If we get it wrong, we’ll ship autonomous systems we can’t explain, can’t reproduce, and can’t safely roll back.

I’m bullish on agents.
I’m just not bullish on agents without flight plans.


## Sources
- [Orchard: An Open-Source Agentic Modeling Framework (Microsoft Research)](https://www.microsoft.com/en-us/research/publication/orchard-an-open-source-agentic-modeling-framework/)
- [Orchard: An Open-Source Agentic Modeling Framework (arXiv:2605.15040)](https://arxiv.org/abs/2605.15040)
- [Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents (project page)](https://ibm.github.io/CLEAR/)
- [Agentic CLEAR (arXiv:2605.22608)](https://arxiv.org/abs/2605.22608)
- [Restricting Drones Near Critical Infrastructure Sites (FAA Newsroom)](https://www.faa.gov/newsroom/restricting-drones-near-critical-infrastructure-sites)
- [Federal Register NPRM via GPO/Justia mirror: FAA proposed rule (May 6, 2026), comments due July 6, 2026](https://regulations.justia.com/regulations/fedreg/2026/05/06/2026-08943.html)