Calibrated achievability
Conformal Prediction
Distribution-free confidence guarantees for every agent decision.
Conformal prediction wraps every agent's output in a mathematically rigorous prediction set. When the system says "90% confident," the true outcome falls within bounds at least 90% of the time, and that holds without assuming your data are Gaussian, that the model is correctly specified, or that the errors behave. It's the calibration layer that makes the urgency × achievability triage in the Decision Stream trustworthy.
Distribution-free is not assumption-free, and we would rather say so than let the phrase do more work than it can. The guarantee rests on exchangeability: that the data it was calibrated on and the thing being predicted come from the same process. A supply chain whose rules genuinely change - a new competitor, a rewritten tariff regime - violates that, and no amount of calibration reaches it.
What the band can do is report the day it stops being valid. The substrate is built to check coverage against realised outcomes on a schedule, so that when the world moves, the promise and the measurement visibly separate - and where there is too little evidence to check, it records that it could not verify rather than staying quiet. A model's own confidence score has no equivalent, because nothing ever checks it.
Conformal prediction delivers valid coverage in finite samples without assuming the shape of the data distribution, the correctness of the model, or well-behaved errors. It assumes one thing: exchangeability between the calibration data and what is being predicted.
"The beauty of conformal prediction is its simplicity and generality. You can wrap any machine learning model in conformal prediction and get valid prediction intervals. No retraining, no distributional assumptions."
Conformal prediction intervals widen when the model is uncertain and narrow when confident, with guaranteed coverage.
Three Approaches to Uncertainty
Not all uncertainty quantification is created equal. Only conformal prediction provides distribution-free guarantees.
Traditional Point Forecast
Approach
Single number, no uncertainty
Output
"Order 500 units"
Failure mode
Breaks when model is wrong
Bayesian Uncertainty
Approach
Requires distributional assumptions
Output
"95% credible interval: 420–580" assuming Gaussian
Failure mode
Breaks when assumptions are wrong
Conformal Prediction
Approach
Distribution-free guarantee
Output
"95% coverage set: 420–580", holds for ANY distribution
Failure mode
Coverage holds even when model is wrong
distribution-free coverage guarantee
No assumptions required
works with any underlying predictor
Model-agnostic
valid with any dataset size
Not just asymptotic
maintains coverage under distribution shift
Non-stationary safe
How It Works
Four steps. No distributional assumptions. Finite-sample coverage guarantee.
Train Any Model
Train any predictive model on historical data. Conformal prediction is model-agnostic, it works with any underlying model.
Measure Prediction Errors
On a calibration set, measure how far off each prediction is. These error scores quantify the model's uncertainty.
Set Coverage Level
Choose your desired coverage level (e.g., 90%). The system determines the corresponding prediction bounds automatically.
Guaranteed Coverage
Every prediction comes with bounds that carry a distribution-free coverage guarantee - a theorem about the method, not a claim about any one dataset. What a deployment then adds is the measurement against its own realised outcomes.
"Conformal prediction is not just an academic curiosity. It's the missing piece that makes AI trustworthy enough for high-stakes deployment, medical diagnosis, autonomous vehicles, and yes, autonomous supply chain decisions."
Why It Matters for Autonomous Decisions
Conformal prediction turns uncertainty from a liability into the governance mechanism behind AI·IO·ML.
Calibrated Confidence Scores
When an agent says "85% likely to prevent stockout," that number is calibrated: across all situations where agents express 85% confidence, outcomes match at least 85% of the time.
Principled Inform Thresholds
Prediction set size feeds the Inform policy. Set size = 1 (one clear action) → agent Automates. Set size > 1 (multiple plausible actions) → agent Informs and the human Inspects. Mathematically grounded governance.
Robust to Distribution Shift
Real operations are non-stationary. Adaptive conformal methods maintain coverage guarantees even as demand patterns, lead times, and supplier behavior change.
"What makes conformal prediction unique is the guarantee. When you say 90% coverage, you mean it, provably, in finite samples. No other uncertainty quantification method can make that claim without distributional assumptions."
guaranteed coverage on demand forecasts
Calibrated intervals
agent Automates autonomously
AI·IO·ML: Automate
agent Informs; human Inspects
AI·IO·ML: Inform & Inspect
Research Foundation
Conformal prediction is backed by decades of peer-reviewed research in machine learning theory, statistical inference, and time-series forecasting.
Vladimir Vovk
Professor, Royal Holloway, University of London
Co-inventor of the conformal prediction framework. Established the theoretical foundations for distribution-free predictive inference.
Glenn Shafer
Professor, Rutgers University
Co-developer of the theory of conformal prediction. Contributed foundational work connecting game-theoretic probability to predictive inference.
Emmanuel Candès
Professor, Stanford University
Pioneered distribution-free inference and conformal methods. Extended conformal prediction to high-dimensional settings and modern statistical problems.
Anastasios Angelopoulos
Assistant Professor, UC Berkeley
Modern conformal prediction for machine learning. Made conformal methods accessible and practical for real-world deployment at scale.
See conformal prediction in action
Watch how calibrated confidence scores govern autonomous agent decisions in real time.