MIT's NANDA initiative put a number on what many operators already suspected: 95% of enterprise generative-AI pilots produce no measurable business impact. The finding has been cited across the industry — usually as an argument about models, vendors, or hype. We read it differently. After a year of building AI systems inside real businesses, we think the number describes an engineering gap, not an AI failure.
The demo and the system are different products
Every failed pilot we've examined follows the same arc. Someone connects a model to a prompt and produces a demo that genuinely impresses — it summarizes the document, drafts the reply, answers the question. Leadership approves a pilot. And then the pilot meets the organization: the CRM that holds the real customer data, the approval chain that governs the workflow, the security team that asks who can see what, the Tuesday-afternoon edge case the demo never saw.
The demo was a product of the model. The system the business needed is a product of engineering — and it was never built. That gap has a consistent anatomy. Five layers, almost always missing:
The five missing layers
1. Integration. A workflow is only automated if it touches the systems where work lives — applications, databases, document stores, email, calendars. Most pilots run beside the business instead of inside it, which means humans become the integration layer, re-keying between the AI and reality. The moment that happens, the pilot costs more than it saves.
2. State and durability. Real workflows pause, wait for approvals, survive failures, and resume. A stateless chat loop can't. Production AI needs durable execution: persistent state, retries, recovery, and the ability to wait three days for a human without losing the thread.
3. Evaluation. Demos are judged by impression; systems are judged by measurement. Without evaluation — what did the AI do, on which inputs, how often was it right, did this version beat the last — a pilot can neither prove its value nor improve. "It seems good" is how pilots die in committee.
4. Human decision points. The teams that succeed don't automate judgment; they automate the work around judgment. Approval gates, escalation paths, and review queues aren't limitations on AI — they're what makes leadership comfortable letting it touch anything important.
5. Security and access. The pilot that ignores access controls, data boundaries, and audit requirements doesn't skip those conversations — it postpones them to the worst possible moment, usually as a veto. Security review belongs in the architecture, not the aftermath.
What the fix looks like
The fix is unglamorous, which is why it's rare: treat production as the requirement from day one. Choose one workflow that matters. Map how it actually runs — including the exceptions. Design the system around the model: integration, state, evaluation, human gates, security. Ship a controlled rollout to real users on real work, measure it, and only then expand.
One workflow in production beats ten demos in a slide deck — because the workflow compounds. It generates evaluation data, trust, and the organizational muscle to deploy the next one. The 95% failed because they tried to prove AI works. The 5% succeeded because they engineered a system that happens to use AI.
A quick self-test
If you have a pilot in flight, five questions predict its fate: Does it read and write to the systems where the work actually lives? Can it pause for a human and resume? Can you say, with numbers, whether last month's version was better? Does anyone know what it's allowed to access — in writing? And is there a named path from pilot to production, with an owner? Two or more "no" answers, and you're on the wrong side of MIT's statistic — not because the AI is weak, but because the system was never built.
Why this keeps happening (it's organizational, not technical)
If the missing layers are well understood — and among production engineers, they are — why do most pilots still skip them? Because of who runs pilots and how they're funded. Pilots typically live with innovation teams or vendors, judged on demonstration speed; production systems live with engineering and operations, judged on reliability. The pilot is optimized to impress in six weeks, so everything that makes production possible — integration plumbing, evaluation harnesses, security review — gets deferred as "phase two." Phase two then requires a different team, a different budget line, and answers to questions the pilot never asked. That handoff is where the 95% dies: not in a dramatic failure, but in a quiet backlog.
The organizations in the successful 5% collapse that handoff. The people who will operate the system are in the room from week one; the security review happens at architecture time; the budget funds a production workflow, not a demonstration. None of this is exotic. It is the same discipline software teams already apply to any system that matters — applied to AI, which somehow gets treated as exempt.
What to do with a pilot that's already stalled
Most companies reading this aren't starting fresh — they have a prototype somewhere between "the demo went great" and "legal has questions." Three honest options. Productionize it: if the workflow is genuinely valuable, the prototype becomes a specification, and the real work begins — building the five layers around the logic the demo proved. Expect the production engineering to be several times the effort of the original prototype; that ratio is normal, not a failure signal. Narrow it: many stalled pilots tried to automate an entire job. Cut scope to the one step with the clearest value — often intake, classification, or first-draft generation — ship that with human review, and expand from evidence. Retire it: some pilots fail the arithmetic — the workflow wasn't expensive enough to justify the system around it. Killing those cleanly is a win; the graveyard of half-alive pilots consumes more credibility than budget.
The wrong option is the common one: leaving the prototype in limbo as proof that "we're doing AI." Limbo pilots quietly teach the organization that AI doesn't work here — which makes the next, better-engineered attempt harder to fund.
The uncomfortable summary
The 95% number gets used to argue that enterprise AI is overhyped. Our experience says something narrower and more fixable: AI capability has outrun AI engineering inside most organizations. The models are ready for far more than they're trusted with; the systems around them mostly don't exist yet. That's not a reason for pessimism — it's a precise description of the work. Companies that do the unglamorous engineering are quietly compounding an advantage while their competitors run demo after demo. The gap between the two groups isn't intelligence or budget. It's whether anyone insisted, on day one, that production was the point.
Where Corinth fits
This is the work we do every day — as fixed-scope engagements and embedded teams. If it sounds like the problem on your desk, talk to an engineer or read how the Production AI Pilot and FDE as a service engagements work.