Why context engineering beats prompt engineering for agents
Most stalled AI projects share the same autopsy. The model works. The demo works. Then it meets real company data and real workflows and falls over. The first instinct is to rewrite the prompt. That is almost never where the problem lives.
Why agents fail on missing information, and the budget decision that follows once you see it.
Short answer: Context engineering is the discipline of giving an AI system the right information at the right moment - the data, tools, and prior steps an agent needs to act. Prompt engineering is wording a single request. For an executive moving from pilots to production, fund context. Agents fail far more on what they can see than on how they were asked.
What is context engineering, in plain terms?
Context engineering is a systems job. You assemble everything an AI needs to see before it acts, and you keep the junk out.
The term comes from Tobi Lütke, Shopify's CEO, who in June 2025 argued it describes the work better than "prompt engineering." He defined it as "the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy backed the term the same week and sharpened it to "the delicate art and science of filling the context window with just the right information for the next step."
That September, Anthropic gave it an engineering definition: "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference." The word doing the work there is scope. The context is the system prompt, the tools the agent can call, the examples you show it, and the running message history, all of it competing for room in a finite window. The instruction you type is a sliver of that.
Why do AI pilots stall when the model is capable?
Pilots stall on integration and workflow gaps. The model is rarely the thing that broke.
The most-cited number of 2025 makes the case. MIT's NANDA initiative found that 95% of enterprise generative-AI pilots deliver no measurable P&L impact, and traced the failures to how the tools connect to real data and workflows rather than to the quality of the models. This is early, directional research. The direction is hard to argue with.
McKinsey finds the same wall in agents specifically. Roughly two-thirds of enterprises have piloted AI agents. Fewer than 10% have scaled one into an actual business function. McKinsey puts the gap down to missing organizational context and data infrastructure rather than a ceiling on what the models can do. The pattern repeats across both studies: companies bought capable tools, wired them into nothing, and watched the pilot stay a pilot.
Why does more context sometimes make an agent worse?
More information is not automatically better. Past a point, extra context drags the answer down. This is exactly why the work is engineering.
Chroma Research tested 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3. Every one degraded in output quality as the input grew, often well before the context window was full. A model sold on a 200,000-token window can lose accuracy at 50,000. They named the effect context rot.
Position matters as much as volume. In Lost in the Middle, researchers from Stanford, UC Berkeley, and Samaya AI showed that models retrieve information reliably when it sits at the start or the end of a long context, and much less reliably when the same fact sits in the middle. Accuracy can swing more than 30% on placement alone. So the job is curation: the right facts, in the right order, with the rest left out. Dumping the whole data lake into the prompt is how you build a slower, more confident wrong answer.
One place you can feel this yourself is AI writing. Give a model the right context and its drafts stop sounding like a model. We packaged ours as a free Claude skill that fixes the sentence patterns that give AI away. Download the human-writing skill.
Is prompt engineering obsolete now?
No. Prompt engineering got demoted to a component of a bigger job.
Wording still counts. A vague instruction produces a vague result, and someone on the team has to write the instruction well. But in Anthropic's framing the prompt is one slice of the context, sitting next to the tools, the retrieved data, and the message history. Sending your staff to a prompt-writing workshop is fine. It does not build the plumbing that decides what an agent knows at the moment it runs, and the plumbing is what the MIT and McKinsey numbers say you are missing. We dug into where wording alone runs out of road in why prompt engineering plateaus.
What should a CEO fund first: prompt training or context infrastructure?
Fund the context infrastructure. A prompt workshop ends by lunch; the context layer is the system your agents run on every day.
The market is already moving here. Gartner frames context engineering as designing the data, workflows, and environment so AI systems act with enterprise-aligned precision without leaning on hand-written prompts. It predicts that by 2028, context-engineering features will be built into 80% of AI-application-building tools and will lift agentic-AI accuracy by at least 30%. The capability is being pushed down into the tooling. The org design around it is still yours to fund.
For an executive, that turns into three budget lines:
- The data layer. Can an agent reach clean, current, permissioned company data at the moment it runs? Most cannot, and that is where the 95% goes to die.
- Retrieval and memory design. Someone has to decide what enters the context window for a given task and what stays out. That is a design job with an owner.
- Accountability. Each agent needs a named human who owns its context and its failures.
Prompt training is a half-day. Those three are the operating system your agents live on. Fund the operating system.
Frequently asked questions
What is context engineering, in plain terms? It is the work of assembling everything an AI needs to see before it acts: company data, the tools it can call, relevant examples, and the running history of the task. Tobi Lütke coined the term in 2025, and Anthropic defines it as curating the optimal set of tokens during inference. The prompt is one part of it.
Is prompt engineering obsolete now? No. Wording a request well still matters, and a vague prompt still produces a vague answer. But prompt engineering is now one input into context engineering, alongside data, tools, and history. Anthropic frames the prompt as a single slice of the context an agent uses. It is a component of the larger job.
How is context engineering different from RAG? RAG (retrieval-augmented generation) is one technique inside context engineering. RAG pulls relevant documents into the prompt at query time. Context engineering is the broader discipline that decides what to retrieve, which tools to expose, how much history to keep, and what to leave out. Think of it as RAG plus everything else that fills the context window.
Why do AI pilots stall even when the model is capable? Because the model was never the constraint. MIT's NANDA initiative found 95% of enterprise generative-AI pilots deliver no measurable P&L impact, traced to integration and workflow gaps. McKinsey reports fewer than 10% of enterprises have scaled an agent into a business function, and points to missing organizational context and data infrastructure. More on that stall pattern in our piece on why AI pilots stall before they scale. The gap is plumbing.
What should a CEO fund first, prompt training or context infrastructure? Context infrastructure. Prompt training is a workshop that ends; context infrastructure is the data access, retrieval design, and accountability an agent runs on every day. Gartner predicts context-engineering features will sit in 80% of AI-app-building tools by 2028 and lift agentic-AI accuracy by at least 30%. Funding the plumbing is what moves a pilot into production.
We built a free Claude skill that makes AI-drafted writing read like a person wrote it, the same context-first thinking applied to the one task every team already runs through AI. Download the human-writing skill. Or, if you just want the next piece on building an AI-native organization, join the newsletter.
Sources
- Tobi Lütke, Shopify CEO, on X (June 19, 2025). x.com
- Andrej Karpathy on X (June 2025). x.com
- Anthropic, "Effective context engineering for AI agents" (September 29, 2025). anthropic.com
- Gartner, "Context engineering: Why it's replacing prompt engineering for enterprise AI success" (2026). gartner.com
- MIT NANDA, "The GenAI Divide: State of AI in Business 2025" (August 2025), via Fortune. fortune.com
- Chroma Research, "Context Rot: How Increasing Input Tokens Impacts LLM Performance" (2025). trychroma.com
- Liu et al. (Stanford, UC Berkeley, Samaya AI), "Lost in the Middle: How Language Models Use Long Contexts" (2023). arxiv.org
- McKinsey, "Reimagining tech infrastructure for agentic AI" (2026); Forbes coverage (March 2026). mckinsey.com / forbes.com
By Christopher Kliebenstein. We build and run AI-native workflows for operators who want working systems over demos.