Evaluation Engineering

Status: draft, revised October 2026. This page states a framing, not a settled position.

The one-sentence version

Evaluation engineering is the design of systems that produce large numbers of estimates and evaluations at known cost, under incentives that keep them honest.

We know how to produce one careful evaluation: hire a capable person, give them time, read the report. We are bad at producing the ten-thousandth as cheaply and reliably as the first, keeping all ten thousand consistent as the world changes, and keeping the system trusted once the evaluated start to care about the scores.

What problem this responds to

Prediction markets, forecasting tournaments and proposals like futarchy looked promising two decades ago. Markets have since become a large business, mostly sports betting, but they and tournaments remain rare as inputs to decisions in government, firms, or even the communities that champion them. One diagnosis is that prediction was never the whole product. Getting from “we can forecast this clean, verifiable question” to “we can put a defensible number on what the decision turns on” needs a structured set of questions (ontology), calculation that chains inputs into derived estimates, prediction to keep the system calibrated, and evaluation to resolve questions with no clean ground truth (The Four Components). Whole machines exist where budgets are large and questions stable: credit scoring, health technology assessment, GiveWell’s cost-effectiveness models. For most grants, organizations, research claims and policies, nobody was building one.

Why “engineering”, and what it has to include

The framing borrows from systems disciplines (lean manufacturing, software architecture) rather than from the philosophy of value or from program evaluation, whose centre of gravity is the individual study (Adjacent Fields §3). The questions are throughput, cost per item, consistency, failure modes, and how a change to one estimate propagates to the estimates that depend on it. At scale, optimizing any single evaluation is usually the wrong objective.

Certification economics complicates this: who pays a certifier shapes what it certifies (Patterns §3). When the catalogue was coded, neither payer type nor method predicted documented failure, but systems paid by the parties they rate had about 3.5 times the odds of one, mostly through gaming by the rated party. The coding is weak evidence (one coder, documented failures only), but a purely technical programme would not have looked. This wiki therefore treats payer, metric ownership, disclosure and appeal as design variables alongside staffing and data, and treats robustness to gaming and legitimacy as properties a design must deliver alongside accuracy and cost (Evaluation as a System). Mechanism design is part of the engineering.

The core split: estimation vs. evaluation

Estimation is producing numbers a careful analyst could check: Fermi estimates, models, sums. Evaluation is judgment on messy questions, where the result is useful only if its audience trusts it (“how much did this organization reduce existential risk?”). The design heuristic is divide and conquer: push as much as possible into cheap, verifiable estimation, and confine judgment-bound evaluation to a smaller, more expensive layer. See Estimation vs. Evaluation.

The core pattern

The technique this wiki leans on most is the prediction–evaluation system: a cheap layer (forecasters, a statistical model, now an LLM) predicts what an expensive, trusted evaluator would conclude on every item; a hidden random subset is sent to the expensive evaluator; the cheap layer’s agreement on that subset is measured against how often two expensive evaluators agree with each other. QURI’s founder wrote this up in 2018, but it was not new. Psychologists showed by 1970 that a statistical model of an expert judge can out-predict the judge, automated essay scoring has run the pattern for years, DARPA’s SCORE program ran it on social-science replication, and prediction-powered inference supplies its statistics (Lineage).

Why evaluation, specifically?

The founding draft’s slogan is that highly optimized evaluations are “(almost) all you need”. Two claims sit behind it:

• Evaluation may be most of the valuable output. Many high-stakes processes are evaluation problems: courts, grantmaking, hiring, impact assessment, policy choice. It is plausible that more resources flow into evaluation than into clean estimation. Nobody has measured this.
• The accuracy bar is lower than it looks. Outputs need only beat what people would otherwise have done, cheaply enough to be used. People already make vast numbers of noisy informal judgments (Objections).

The slogan is the least-supported claim in the wiki and nothing else depends on it.

Not AI model evals, though they overlap

In current AI usage, “evals” means test suites and judges that measure AI models and AI products: capability and safety evaluations by groups such as METR, whose time-horizon metric tracks the length of tasks models can complete (Kwa et al. 2025); open frameworks such as the UK AI Security Institute’s Inspect; and rubric-based testing of LLM applications. That field is large and well funded. This wiki uses “evaluation” in the older sense of judging grants, organizations, policies and research, where an AI is a possible evaluator rather than the thing evaluated.

The two meet wherever an LLM is the judge. Validating a judge against humans, putting error bars on results (Miller 2024), and watching a leaderboard get gamed (selective disclosure of privately tested model variants biased Chatbot Arena scores; Singh et al. 2025) are shared problems, and the LLM chapter and Measuring an Evaluator draw on the AI-evals literature for them. Whether the name should change is crux 12.

Why this might be worth a field

• Generality. Better evaluation systems would help impact assessment, policy, research prioritization and grantmaking at once.
• Neglect, partly. The components (forecasting, decision analysis, survey methodology, LLM judging) are each studied. Their integration into audited, high-throughput systems for judging grants, organizations and policies is not, as far as this wiki can tell. The AI-evals boom weakens this claim for anything that looks like judging model outputs.
• Timing. LLMs make the labor-intensive steps (drafting evaluations, structuring questions, running calculations) cheap. Whether they make them trustworthy is the open question (LLMs as Evaluators).

What this wiki is not yet

It does not commit to a software stack, an institutional form, or claims about cost-effectiveness; those depend on experiments not yet run. Its capability ladder is a v0 graded on eight systems, and its one fully priced design is the fictional worked example.