Harness Engineering

Summary

Harness engineering is a mental model for increasing the reliability and autonomy of AI coding agents by implementing an "outer harness" around the underlying model. This approach uses a combination of feedforward guides and feedback sensors to reduce human review toil and enable agents to self-correct without constant supervision.

Key Points

• The fundamental equation for an AI agent is defined as: Agent = Model + Harness.
• Harnesses consist of two primary control mechanisms: Guides (feedforward controls that steer behavior before action) and Sensors (feedback controls that observe and enable self-correction after action).
• Controls are categorized by execution type: Computational (deterministic, CPU-based, e.g., linters, type checkers, and unit tests) and Inferential (non-deterministic, GPU/NPU-based, e.g., semantic analysis and "LLM as judge").
• Harnessing is applied across three regulatory dimensions: Maintainability (code quality and complexity), Architecture Fitness (structural constraints and performance requirements), and Behaviour (functional correctness).
• Harnessability is a property of the codebase; strongly typed languages and well-defined module boundaries increase the ease of implementing effective sensors and guides.

Technical Details

The harness functions as a cybernetic governor through a continuous "steering loop." Developers iterate on the harness by using computational sensors—such as ArchUnit for structural boundary checks or OpenRewrite for automated code modifications—to provide fast, deterministic feedback. These are complemented by inferential sensors that utilize LLMs to perform semantic analysis, such as identifying redundant tests or detecting architectural drift, though these are more computationally expensive and non-deterministic.

Effective implementation requires "shifting left" by distributing controls across the development lifecycle based on cost and latency. Low-latency computational controls (e.g., linters and fast test suites) should be executed pre-commit or during the initial agent interaction. Higher-latency, expensive inferential controls (e.g., mutation testing or broad-scope code reviews) are better suited for post-integration pipelines. The ultimate goal is to create a system where the agent can monitor runtime feedback, such as degrading SLOs or log anomalies, to suggest proactive improvements.

Impact / Why It Matters

Implementing a robust harness reduces the cognitive load on developers by automating the detection of errors and architectural violations. It shifts the developer's role from manual code reviewer to the architect of the agent's steering loop and regulatory framework.