§ Predict · Methodology

How forecasts work.

Origin

Most forecasts emerge from the council pass on a published signal: a Forecaster agent projects 2nd-order effects, names a confidence band, and sets a resolution target. A subset comes from research investigations on user prompts, where the runner produces a forecast as part of the briefing. Council-auto creation triggers when a signal's heat (combination of impact and corroboration) crosses a threshold; in that case the desk reviews before publish.

Schema

Each forecast carries:

  • statement — the claim, in present tense.
  • confidence — point estimate, 0-1, never 0 or 1.
  • confidence_band_low/_high — calibrated bounds.
  • resolution_target_date — when the resolver checks.
  • resolution_criteria_md — what counts as a hit.
  • driving_signal_ids — every signal that supports it.
  • counterview_md — the strongest case against.
  • track_record_weight — 1.0 by default, lower for noisy categories.

Resolution

A nightly CronWorkflow walks every open forecast past its target date and asks Opus 4.8 to compare the criteria against subsequent signals. The worker writes one of:

  • hit — criteria met
  • miss — criteria explicitly not met
  • partial — criteria met partially or with caveats
  • cancelled — facts on the ground made the criteria moot (e.g. brand was acquired, geography changed)
  • superseded — a later forecast replaces it (linked via superseded_by_id)

The narrative for each resolution is published on the forecast detail page.

Calibration

The calibration table on track record shows realized hit rates per confidence decile. A well-calibrated desk shows realized rates close to the midpoint of each band. The desk targets calibration over hit count.

Honesty constraints

  • The statement is immutable post-publish (DB trigger enforces).
  • confidence < 1.0 (CHECK constraint).
  • Misses are not hidden. The track-record page includes them by default.
  • Revisions create forecast_revisions rows; the original is preserved.