From Prompt Engineering to Harness Engineering

Prompt engineering made sense when the unit of work was one answer.

You wrote a better instruction, the model produced a better completion, and the whole interaction ended inside the chat box. That was useful. It was also a tiny abstraction over autocomplete: one request, one response, maybe a little tool use if the interface was fancy.

That is not the shape of frontier coding agents anymore.

The interesting primitives in Codex and Claude Code are not “make the next answer nicer.” They are ways to keep an agent working inside a bounded environment until the work is proven: Codex /goal, Claude Code ultracode, Dynamic Workflows, subagents, hooks, repo instructions, sandboxes, eval loops, progress logs, and review packets. The prompt still matters, but it is no longer the whole interface. It is the seed for a harness.

OpenAI has already named the shift. In “Harness engineering: leveraging Codex in an agent-first world”, Ryan Lopopolo describes an internal experiment where a small team built and shipped a real software product with Codex writing the code, docs, tests, CI, observability, tooling, and pull requests. The important line is not that the agents wrote a lot of code. The important line is the operating principle: “Humans steer. Agents execute.”

That is the move from prompt engineering to harness engineering.

The lab primitives are converging

The two frontier coding-agent leaders are approaching the same bottleneck from different sides.

OpenAI’s key primitive is Codex /goal. The official Codex goal-mode guide says to use /goal when a task needs Codex “to keep working across turns toward a verifiable stopping condition.” The page is blunt about the useful part: “Codex should know what ‘done’ means before it starts.”

Anthropic’s key primitive is Claude Code ultracode, backed by Dynamic Workflows. The Claude Code model configuration docs describe ultracode as a setting that plans a Dynamic Workflow for substantive tasks with extra-high reasoning. The Dynamic Workflows docs describe the underlying mechanism: Claude writes a JavaScript orchestration script, the runtime executes it in the background, and many subagents can work under that script while the main session stays manageable.

Those are different product shapes, but the underlying pattern is the same:

goal
→ context
→ action
→ observation
→ verification
→ correction
→ stop or escalate

A prompt asks for a result. A harness creates the conditions where an agent can pursue the result, inspect reality, correct course, and stop for a reason other than “the assistant wrote a confident paragraph.”

Codex /goal is the depth loop

Codex /goal is the cleanest version of a depth loop: one durable objective, many turns, repeated checks.

The bad version is this:

/goal improve the app

That sounds autonomous, but it is mostly a permission slip for drift. “Improve” has no boundary. The agent can change the styling, rewrite a component, add tests, delete a feature, or spend two hours polishing the wrong surface and still narrate the ending like it did you a favor.

The useful version looks more like this:

/goal migrate the billing webhook handler to the new event schema without changing public API behavior. Stop only when unit tests, integration tests, and replay fixtures pass. If replay parity fails twice, pause and report the mismatch.

That is not a longer prompt for its own sake. It is a contract:

• one objective
• explicit non-goal
• verification commands
• retry behavior
• stop/pause condition

The Codex best practices page gives almost the same shape as a default prompt structure: goal, context, constraints, and “done when.” That is the useful bridge. The old prompt-engineering instinct says to keep adding instructions. Harness engineering says to externalize the pieces that matter: put repo rules in AGENTS.md, point at files instead of pasting them, define checks the agent can run, and make completion depend on evidence.

OpenAI’s more advanced iterate-on-difficult-problems guide makes the loop explicit: inspect the current output, score it, identify the next improvement, make one focused change, rerun the evaluation, and continue until the target is reached. That is not magic. It is hill-climbing with a measurement surface.

The measurement surface is the part people skip, because writing evals and fixtures is less fun than asking the model to be smarter. Unfortunately, that is exactly why it matters. Without a check, /goal is just a longer leash.

Claude ultracode is the width loop

Claude Code’s ultracode points in a different direction. It is not just “think harder.” The interesting part is that Claude can decide when a substantive task should become a Dynamic Workflow.

A Dynamic Workflow moves orchestration out of the chat transcript and into code. Instead of one model dragging every intermediate result through one context window, Claude can write a workflow script, spawn subagents, collect their results, cross-check findings, and return a consolidated answer.

The rough shape is:

plan
→ split
→ spawn subagents
→ collect results
→ cross-check
→ integrate
→ verify

That is a width loop. It is useful when the task has independent branches: a repo-wide audit, a migration across modules, a security review, a multi-source research pass, or a planning problem where separate agents should argue from different evidence.

The power is also the risk. If Codex /goal can turn a vague objective into a meandering depth loop, ultracode can turn a vague objective into a very expensive committee. More agents do not automatically mean more rigor. Sometimes it means five context windows confidently sharing the same bad premise.

Anthropic’s older Building effective agents guidance still applies: add complexity only when it demonstrably improves outcomes. Dynamic Workflows should buy one of three things:

1. Parallel evidence — independent investigations that would be slow sequentially.
2. Independent verification — a reviewer or grader that does not share the maker’s context.
3. Manageable orchestration — a script or artifact that keeps intermediate state out of the main chat.

If the workflow only produces more prose, it is not harness engineering. It is process cosplay with a token budget.

Planning did not disappear

The easy wrong takeaway is that loops replace specs.

They do not. They replace over-specified procedural prompts.

The agent can choose the path. It cannot choose what “done” means. That boundary is the whole job.

A good harness contract is smaller than a giant spec, but stronger than a vibe:

Goal:
What must be true when this is done?

Non-goals:
What should not change?

Context:
Which files, docs, issues, logs, screenshots, or plans matter?

Allowed actions:
What can the agent edit, run, spawn, or access?

Verification:
Which tests, evals, screenshots, reviewers, or artifacts prove success?

Budget:
How many iterations, agents, minutes, failed checks, or tokens before pausing?

Escalation:
When should the agent stop and ask?

Artifacts:
Where should plans, logs, diffs, scores, and decisions be written?

This is why OpenAI’s Codex article is useful beyond Codex. The team did not just “prompt harder.” They made the repository legible to agents. They created scaffolding, repeatable workflows, validation systems, agent-readable docs, PR review loops, and feedback mechanisms. When Codex failed, the question became: what capability is missing, and how do we make it legible and enforceable for the agent?

That is a different engineering posture.

Prompt engineering asks: “What words will make the model behave?”

Harness engineering asks:

• What state should live outside the model?
• What context should the agent retrieve instead of remember?
• What tools can it safely use?
• What should block completion?
• What should be logged for review?
• What should become repo guidance after the second repeated mistake?
• When is a subagent worth the cost?
• When should the loop stop?

The prompt is still there. It is just no longer load-bearing by itself.

Verifiers matter because self-critique is cheap theater

The most important caveat in Lance Martin’s thread about getting more from frontier models is not “use more agents.” It is that the judging context matters. He writes that a verifier subagent tends to outperform self-critique because grading happens in an independent context window.

That matches the practical failure mode anyone using coding agents has seen: the agent that made the change is very good at explaining why the change is probably fine. It has already paid the cognitive cost of the approach. It has local momentum. It has a transcript full of its own justifications.

A verifier should not be another ceremonial paragraph. It should have a narrower job:

maker agent: implement the change
hard gate: run deterministic tests/evals
verifier agent: inspect only the diff, logs, and rubric
human: review exceptions, not every step

Codex supports subagent workflows for parallel review and exploration. Claude Code Dynamic Workflows can make verifier agents part of the orchestration. OpenAI’s internal Codex practice shifted much of the review burden toward agent-to-agent review. Anthropic’s long-running-agent harness guidance points at the same need: make progress inspectable across sessions, keep feature lists and logs, and require end-to-end testing before marking work complete.

The pattern is not “trust the agent more.” It is “give the agent enough rope, then make the harness count the rope.”

Safety is also harness engineering

Harness engineering is not only about going faster. It is also how frontier agents become acceptable inside real workflows.

OpenAI’s Running Codex safely at OpenAI describes the control surfaces directly: sandboxing, approval policies, network controls, credential management, command-level rules, telemetry, and audit logs. That is not separate from productivity. It is what lets an agent keep working without asking a human about every boring safe action or silently doing the dangerous ones.

Claude Code has the same class of surfaces: permissions, hooks, settings scopes, subagents, skills, project instructions, and workflows. Hooks can validate or block tool calls, inject context, run tests, log activity, and automate review gates. Again, the point is not a better incantation. The point is shaping the environment where the model acts.

The harness is where taste becomes infrastructure.

The issue tracker becomes the control plane

The mature endpoint is not one heroic terminal session. It is work organized around durable tasks.

OpenAI’s Symphony spec pushes Codex orchestration into the issue tracker: every open task gets an agent in its own workspace, agents run until the task reaches a handoff state, stalled work gets restarted, CI gets watched, flaky checks get retried, and humans review artifacts instead of babysitting terminals.

That is the real end of prompt engineering as the main interface. The unit of control becomes the ticket, branch, workflow, test, and review packet. The prompt becomes one input into a larger system that remembers what is open, what is blocked, what passed, what failed, and what should happen next.

This is also where Claude Dynamic Workflows make conceptual sense. A workflow script is a temporary control plane for a complex task. Symphony is a persistent control plane for a team’s task graph. Same direction, different scope.

The new skill is deciding what gets externalized

The practical advice is not “use /goal for everything” or “turn on ultracode and walk away.” That is how you get a very expensive pile of almost-correct work.

Use Codex /goal when the work is coherent enough to be one objective and has a clear stop condition:

• migrate this subsystem
• get this test suite green
• implement this plan until the acceptance checks pass
• retry this deploy until health checks pass or the same failure repeats

Use Claude ultracode or an explicit Dynamic Workflow when the work benefits from width:

• audit these modules independently
• compare implementation strategies
• review a large diff from multiple angles
• gather evidence across many files or sources
• split a migration into independent branches and converge

Use neither when the task is a one-line change. A normal prompt is still fine when the work is small, local, and easy to verify. Harness engineering is not a license to build a cathedral around a typo.

The shift is knowing when the prompt should become a loop, and what proof the loop owes you before it stops.

The prompt becomes the seed

Prompt engineering got us better answers.

Harness engineering gets us agents that can keep working without losing the plot.

That does not mean specs vanish. It means the spec changes shape. Instead of one giant instruction blob trying to control every step, the work is split across the environment: durable repo instructions, scoped goals, explicit non-goals, tool permissions, tests, evals, verifier agents, workflow scripts, progress logs, and stop conditions.

The frontier labs are already shipping the pieces. Codex /goal gives the depth loop. Claude ultracode gives the width loop through Dynamic Workflows. Both are pointing at the same bottleneck breaker.

The prompt is no longer the system.

The harness is.