Aug 2026
6 min read
Probabilistic Modelling
From EMA to Bayesian capability inference
Measuring belief states under uncertainty
A first exploration into sequential inference. Examining how Exponential Moving Averages collapse observations into a single estimate, and how Beta-Binomial updating instead maintains an explicit belief distribution and uncertainty as evidence accumulates.
§01 · The Limitation of Point Estimators
Most scoring systems summarize a series of evaluations into a single moving scalar. A classic example is the Exponential Moving Average (EMA), which updates a capability estimate after each observation.
While lightweight, EMA suffers from an inherent epistemic flaw: it treats each event with a fixed learning rate and discards the historical sample count. It cannot differentiate between a 65% score computed from 3 observations and a 65% score confirmed across 300 evaluations.
θ_{t+1} = θ_t + η(y_t - θ_t)EMA update: produces a point estimate without uncertainty representation.
§02 · Conjugate Beta-Binomial Updating
To preserve uncertainty, we shift from a single number to a probability density over capability. By modeling capability θ as a continuous parameter on [0, 1] with a conjugate Beta prior, each binary task outcome y ∈ {0, 1} yields an exact closed-form posterior update.
The system initializes with an open prior Beta(1, 1) (uniform uncertainty). For each success, α increments by 1; for each failure, β increments by 1. The density naturally concentrates as evidence accumulates.
p(θ | s, f) ∝ θ^{(α + s) - 1} (1 - θ)^{(β + f) - 1} = Beta(α', β')Bayesian posterior update: α' = α + s, β' = β + f
- ›Belief (Expected Value): E[θ] = α / (α + β)
- ›Uncertainty (Variance): Var(θ) = (α · β) / [(α + β)² · (α + β + 1)]
- ›Evidence (Sample Depth): N = s + f
§03 · The 9-Observation Mixed-Evidence Experiment
In our baseline validation experiment, the system observed 9 sequential outcomes (6 successes, 3 failures).
Starting from Beta(1, 1), the resulting posterior density became Beta(7, 4). The expected capability is 0.6364 with a standard deviation of ± 0.1389 and a 90% credible interval spanning [0.395, 0.852].
“The crucial achievement is not the number 0.6364, but the fact that the system explicitly quantifies how much it still does not know.”