thesis institute
Agents & packs

The forecaster, and how expertise reaches it

Brier-1 writes the Institute's forecasts — every prediction published with its reasoning and scored when reality arrives. Packs are how human expertise enters that loop: small, versioned, falsifiable units of judgment that have to improve the forecasts to earn their place, and are measured against reality once they do.

Contribution
Pack

An expert's encoded judgment about one class of question.

Agent
Brier-1

Composes the applicable packs into its reasoning and predicts.

Output
Forecast

Published with its reasoning and the packs that shaped it.

Reality
Score

Resolved and Brier-scored — credit flows back to the packs.

The idea

A pack is a falsifiable claim

Most forecasting platforms aggregate predictions. The Institute aggregates judgment — and scores it. A pack is an expert asserting “this way of thinking about this kind of question makes the forecasts better,” in a form the system can test, version, attribute, and retire. It is not a contribution of text. It is a wager that has to win on a scoreboard.

If expertise transfers, it shows up as a lower Brier score. If it doesn't, the pack doesn't get in — and saying so is the whole point.

Anatomy

What's inside a pack

A pack manifest carries five kinds of judgment, plus the record that admitted it:

  • Evidence to consider
    Which sources to pull — each pinned to a snapshot, so “the evidence changed” is detectable, not silent.
  • How to weigh it
    An explicit protocol: base rates first, likelihood ratios by evidence type, recency discounting.
  • Mechanism models
    Theoretical templates the agent must instantiate before it reaches for a number.
  • Computational models
    Tools the agent actually runs — a policy pack invokes PolicyEngine or the Axiom rules engine instead of guessing.
  • Correlation directives
    How this question covaries with others, so related forecasts are scored on joint calibration, not one at a time.
Two kinds

Judgment, and computation

Judgment pack

Encoded reasoning

The transferable habits of a good forecaster — base-rate anchoring, mechanism decomposition, stating an interval before a point estimate, a premortem. Judgment that usually lives in one expert's head, written down where it can be reused and tested.

e.g. “evidence-weighting protocol” — the Institute's own forecasting method, packaged.
Computational pack

Grounded in a model

For questions a model can actually compute, the pack hands the agent the model. A policy-outcome pack runs PolicyEngine or the open Axiom rules engine — deterministic eligibility and benefit math, traceable to the statute — so a “what will this reform cost / who qualifies” forecast rests on computation, not a hunch.

e.g. “SNAP policy grounding” — eligibility computed, not estimated; even client-side, no data leaving the page.
Admission

How a pack earns its place

Anyone can propose a pack. Whether it gets used is decided by a two-tier gate — cheap checks on every change, and an expensive verdict against resolved questions.

Every change · deterministic

Fast tier

  • The manifest validates and its scope is well-formed.
  • Required reasoning blocks are present and ordered — interval before point estimate.
  • Anti-sycophancy framings are intact.
  • Every declared tool is actually invocable.
On demand & scheduled · against reality

Slow tier — replay with ablation

  • Replay a frozen set of resolved questions through Brier-1, with and without the pack.
  • Compute the Brier-score and calibration delta the pack is responsible for.
  • Admit it only if it improves — or holds — its claimed question class.
  • Ablation gives each pack its marginal credit when several apply at once.
Goodhart defense

Scoring runs on a time-split holdout — train on questions resolved before a cutoff, score on questions resolved after. A pack tuned to look good on the past can't quietly overfit the test; it has to generalize forward, which is the only thing a forecast cares about.

Provenance & credit

A citation system where the citation is a score

Every forecast records the packs and versions that shaped it. Calibration can then be segmented by pack generation, and an expert's contribution becomes a measurable number: the Brier delta their pack moved. Credit isn't a byline — it's an effect on accuracy.

Pack · versionBrier ΔStatus
snap-policy-grounding 0.3.0
−0.041
admitted
base-rate-first 1.2.0
−0.018
admitted
election-mechanism 0.1.0
+0.002
not retained

Illustrative figures. A pack that stops helping — because the world moved or its evidence went stale — is retired the same way it was admitted: by the score, in the open.

Composition

How packs reach the prediction

Brier-1 reasons through a fixed elicitation flow. Applicable packs compose into the relevant steps — they refine the method, they don't replace it.

frame the question base rates mechanism weigh evidence interval before point premortem correlation

And when a question can't be honestly answered, the agent has to say so — with a typed escape, never a confident guess. That refusal is the framework's thesis in miniature.

KPI unresolvable as stated no defensible base rate question ill-posed
Contribute

Bring a hypothesis

If you think a kind of evidence, a model, a mechanism, or a piece of open software would make the forecasts better, that's a pack. Write it down, and let the scoreboard decide. The whole system is open — the agent, the packs, the gate, and the record.

Brier and its pack system are open source, of a piece with the rest of the stack — open source, open data, open weights, open predictions.