Forecast archive
Each eligible forecast should be retained with its publication date, forecast horizon and original output.
RATE CHALLENGE MODEL PERFORMANCE
Model Performance
A permanent record for assessing how Rate Challenge forecasts perform over time — including dated forecasts, actual outcomes, calibration and methodology changes.
Dated forecasts retained
Forecast vs actual
Calibration reviewed
Method changes disclosed
No Rate Challenge forecast series has yet accumulated enough published history for meaningful performance statistics.
This page establishes the performance framework now so future results can be evaluated against a consistent standard.
Performance framework
How a forecast will be judged
Forecast quality is not simply whether one headline prediction was right. A useful assessment considers probability calibration, range coverage, directional accuracy, timing and whether the model behaved consistently across different conditions.
Actual outcome
Once the forecast horizon passes, the realised outcome can be recorded alongside the original forecast.
Calibration
If events assigned a 70% probability occur roughly seven times in ten over a suitable sample, that is evidence of useful calibration.
Error and coverage
Where models publish ranges or numerical estimates, error size and whether actual outcomes fell within the forecast range can be reviewed.
What will be tracked
Different model types need different performance measures
A cash-rate probability model should not be judged exactly the same way as a property-price range model. The measures below describe the types of checks that may be used where they suit the forecast.
Probability calibration
Whether events occurred at rates broadly consistent with the probabilities assigned to them.
Directional accuracy
Whether the model correctly identified the broad direction of movement where direction was part of the forecast.
Range coverage
How often actual outcomes fell within published forecast ranges.
Forecast error
The size of the difference between numerical forecasts and realised outcomes where a point estimate is appropriate.
Timing accuracy
Whether predicted changes occurred within the expected time window.
Benchmark comparison
Where suitable benchmarks exist, model performance may be compared with simple alternatives or external reference forecasts.
Permanent historical record
Old forecasts should not disappear when the outlook changes
Forecasts inevitably change as new information arrives. Updating a model is appropriate; overwriting the historical record is not. The performance framework is designed so dated outputs can remain available for later comparison.
- Original publication date retained
- Original probability or range retained
- Outcome recorded after the horizon passes
- Material methodology changes dated
Current status
Framework established
Performance statistics will populate as published forecast history becomes available.
Forecast archiveTo populate
Performance historyTo populate
Interpretation
01
02
03
04
What strong model performance does — and does not — mean
No model is certain
Good historical performance does not make a future forecast guaranteed.
Sample size matters
A handful of forecasts is not enough to make strong claims about long-run model quality.
Conditions change
Relationships that worked in one period may weaken when policy, markets or economic conditions change.
Method changes matter
Performance should be interpreted alongside material changes to data, assumptions or modelling methods.
Related research
Explore the forecast and data framework
Model performance FAQs
How to read future performance results
Why publish a Model Performance page before there is a long forecast history?
Creating the framework first makes the evaluation standard clear before results are known. That reduces the temptation to choose performance measures after seeing the outcomes.
Does one correct forecast prove a model works?
No. Individual forecasts can be right or wrong by chance. Meaningful performance assessment requires a suitable history and should consider calibration, error, coverage and the conditions in which forecasts were made.
What is probability calibration?
Calibration asks whether events occur roughly as often as their assigned probabilities suggest over a suitable sample. For example, outcomes given similar high probabilities should occur more frequently than outcomes given low probabilities.
Will poor forecasts remain visible?
The intended framework is to preserve dated forecast outputs so both strong and weak results can be assessed after outcomes are known.
Is model performance a guarantee of future accuracy?
No. Historical performance is useful evidence, but future conditions can differ and all forecasts remain uncertain.
Transparency over hindsight
Judge the forecast against the record.
As the Rate Challenge model library grows, this page will become the central record for historical forecast performance.