2024-2025 NCAAM Model Performance Analysis
Scope
All scored games in the selected league and season. AP Poll is excluded here.
Season
Comparing prediction accuracy across 1571 games using multiple rating models.
7-day holdout coverage: 6/17 models .
Rolling Holdout Curves
Each point is a strict weekly holdout: train on all games before that week, test on that week. This first version uses a 21-day warmup, then 7-day holdouts stepped forward weekly.
No rolling weekly validation rows are stored for this league/season yet.
Model Performance Leaderboard
Models ranked by strict holdout AUC when available (fallback: full-season AUC). Hover over column headers for explanations.
| # | Model | 7d Split | AUC | Acc | Brier | LogLoss | n | AUC 7d | Acc 7d | Brier 7d | n 7d |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Adjusted Efficiency Adjusted Efficiency Opponent-adjusted efficiency model with separate offensive and defensive components. More → |
FULL
no 7d
|
0.870 | 77.9% | 0.146 | 0.436 | 2212 | - | - | - | 0 |
| 2 | Log Adjusted Log Adjusted Log-scale adjusted efficiency model that downweights blowout leverage. More → |
FULL
no 7d
|
0.870 | 77.8% | 0.147 | 0.437 | 2212 | - | - | - | 0 |
| 3 | Efficiency Efficiency Tempo-adjusted efficiency version of Pythagorean ratings. More → |
FULL
no 7d
|
0.808 | 72.8% | 0.188 | 0.583 | 2212 | - | - | - | 0 |
| 4 | Margin Margin Linear team-strength model fit on point differential instead of binary wins. More → |
STRICT
226g
|
0.829 | 73.7% | 0.171 | 0.512 | 2212 | 0.750 | 66.8% | 0.205 | 226 |
| 5 | Elo Elo Streaming paired-comparison rating with recency baked into sequential updates. More → |
STRICT
226g
|
0.758 | 69.5% | 0.197 | 0.573 | 2212 | 0.680 | 65.9% | 0.229 | 226 |
| 6 | Bradley-Terry Bradley-Terry Static logistic paired-comparison model with one team strength parameter. More → |
STRICT
226g
|
0.841 | 75.8% | 0.167 | 0.504 | 2212 | 0.678 | 62.8% | 0.228 | 226 |
| 7 | Avg Margin Baseline Avg Margin Baseline Predict from simple average scoring margin in the training window. More → |
STRICT
226g
|
0.800 | 72.7% | 0.182 | 0.539 | 2212 | 0.675 | 61.9% | 0.226 | 226 |
| 8 | Pythagorean Pythagorean Pythagorean win expectation from raw points scored and allowed. More → |
STRICT
226g
|
0.833 | 75.6% | 0.173 | 0.520 | 2212 | 0.661 | 59.7% | 0.242 | 226 |
| 9 | Home Team Baseline Home Team Baseline Always favor the home team with a fixed prior. More → |
STRICT
226g
|
0.689 | 69.5% | 0.221 | 0.634 | 2212 | 0.584 | 58.4% | 0.243 | 226 |
| - | Bradley-Terry Recency Bradley-Terry Recency Static Bradley-Terry with exponential recency weights on newer games. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Margin Recency Margin Recency Margin regression with exponential recency weights on newer games. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Points Off/Def Points Off/Def Raw points regression with separate offensive and defensive team parameters. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Points Off/Def Recency Points Off/Def Recency Off/def points regression with exponential recency weights. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Core Ensemble Core Ensemble Equal-logit blend of Elo, recency BT, recency margin, log-adjusted pyth, and points off/def. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Recency Ensemble Recency Ensemble Equal-logit blend of Elo, recency BT, recency margin, log-adjusted pyth, and recency points off/def. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Dynamic Bradley-Terry Dynamic Bradley-Terry Time-evolving paired-comparison model with latent team strength drift. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
| - | Adjusted Context Blend Adjusted Context Blend Experimental context-heavy win model blending strong team components with rest and venue context. More → |
FULL
no 7d
|
- | - | - | - | 0 | - | - | - | 0 |
Methodology
ELO / Bradley-Terry
- ELO: Iterative updates, K=64, HCA=100
- BT: Static logistic regression on all games
- Both model win probability, not margin
- ELO updates after each game; BT fits once
Pythagorean Models
- Raw: Classic points scored/allowed formula
- Efficiency: Pace-adjusted (pts per possession)
- Adjusted: Opponent-adjusted efficiency
- Log: Log-linear multiplicative scale
Margin Regression
- Team-level ridge regression on point margin
- Linear Bradley-Terry (margin target)
- Alpha=0.05 (CV-tuned)
- Learns home advantage from data (~6 pts)
Baselines
- Home Team: Always predict home wins (60%)
- Avg Margin: Higher average margin wins
- Models should beat these to add value