2014-2015 NCAAW Model Performance
Scope
All scored games in the selected league and season. AP Poll is excluded here.
Season
Every ranked model predicted the same 4106 games (2014-12-10 to 2015-04-07, 17 weekly windows) out of sample: each week's games were predicted by models fit only on the season's earlier games.
Does recruiting help team predictions?
Women: not measured yet. The women's preseason estimate has no recruiting
data; its roster part alone beats carrying last season over by 40 milli-nats per game in
weeks 1-4 and 14 over the full season.
Milli-nats: 0.001 of log loss per game, lower is better.
Recruiting model report
· Preseason prior backtest
Model Performance Leaderboard
Ranked by full-season AUC. The final week is the season's last 7 played days (6 games, 2015-04-01 to 2015-04-07), predicted from every earlier game: a small sample that shows the latest form.
| # | Model | AUC | Acc | Brier | LogLoss | AUC final wk | Acc final wk |
|---|---|---|---|---|---|---|---|
| 1 | Margin Recency Margin Recency Margin regression with exponential recency weights on newer games. More โ | 0.850 | 76.0% | 0.163 | 0.495 | 0.750 | 66.7% |
| 2 | Margin Margin Linear team-strength model fit on point differential instead of binary wins. More โ | 0.849 | 75.6% | 0.163 | 0.496 | 0.750 | 66.7% |
| 3 | Points Off/Def Recency Points Off/Def Recency Off/def points regression with exponential recency weights. More โ | 0.848 | 75.7% | 0.165 | 0.501 | 0.750 | 66.7% |
| 4 | Points Off/Def Points Off/Def Raw points regression with separate offensive and defensive team parameters. More โ | 0.847 | 75.7% | 0.165 | 0.502 | 0.750 | 66.7% |
| 5 | Adjusted Context Blend Adjusted Context Blend Chronologically trained component blend with rest and signed venue context. More โ | 0.841 | 75.6% | 0.166 | 0.503 | 0.750 | 66.7% |
| 6 | Recency Ensemble Recency Ensemble Equal-logit blend of Elo, recency BT, recency margin, log-adjusted pyth, and recency points off/def. More โ | 0.840 | 75.3% | 0.164 | 0.495 | 0.750 | 66.7% |
| 7 | Core Ensemble Core Ensemble Equal-logit blend of Elo, recency BT, recency margin, log-adjusted pyth, and points off/def. More โ | 0.840 | 75.3% | 0.164 | 0.495 | 0.750 | 66.7% |
| 8 | Adjusted Efficiency Adjusted Efficiency Opponent-adjusted efficiency model with separate offensive and defensive components. More โ | 0.832 | 74.6% | 0.173 | 0.530 | 0.750 | 66.7% |
| 9 | Log Adjusted Log Adjusted Log-scale adjusted efficiency model that downweights blowout leverage. More โ | 0.831 | 74.7% | 0.174 | 0.532 | 0.750 | 66.7% |
| 10 | Elo Elo Streaming paired-comparison rating with recency baked into sequential updates. More โ | 0.821 | 74.4% | 0.173 | 0.518 | 0.500 | 66.7% |
| 11 | Bradley-Terry Recency Bradley-Terry Recency Static Bradley-Terry with exponential recency weights on newer games. More โ | 0.815 | 73.6% | 0.177 | 0.531 | 0.500 | 66.7% |
| 12 | Bradley-Terry Bradley-Terry Static logistic paired-comparison model with one team strength parameter. More โ | 0.814 | 73.6% | 0.178 | 0.532 | 0.500 | 66.7% |
| 13 | Dynamic Bradley-Terry Dynamic Bradley-Terry Time-evolving paired-comparison model with latent team strength drift. More โ | 0.807 | 73.0% | 0.180 | 0.536 | 0.375 | 50.0% |
| 14 | Avg Margin Baseline Avg Margin Baseline Predict from simple average scoring margin in the training window. More โ | 0.800 | 72.6% | 0.183 | 0.544 | 0.625 | 50.0% |
| 15 | Pythagorean Pythagorean Pythagorean win expectation from raw points scored and allowed. More โ | 0.793 | 73.7% | 0.191 | 0.564 | 0.375 | 50.0% |
| 16 | Home Team Baseline Home Team Baseline Always favor the home team with a fixed prior. More โ | 0.614 | 61.4% | 0.237 | 0.667 | 0.625 | 66.7% |
Methodology
Out-of-sample windows
- The season is split into 7-day windows ending on its last played day
- Each window is predicted by models refit on all earlier games of the season
- The first window starts after 21 days of games
- Every model is scored on the same games; a model that cannot predict most of them is listed as not compared
Elo / Bradley-Terry
- Elo: the published league Elo (preseason priors and carryover where configured), frozen at the window start
- Bradley-Terry: static logistic fit on the earlier games
- Recency variants weight games with a 120-day half-life in NCAA leagues (keeping the non-conference games that link conferences) and a 30-day half-life in pro leagues
Pythagorean and margin models
- Raw, adjusted and log-adjusted Pythagorean from scores; efficiency from box-score possessions
- Margin: ridge regression on point margin with a learned home advantage
- Ensembles average the log-odds of five families
Baselines
- Home Team: the home team wins 60% (50% at neutral sites)
- Avg Margin: the higher average margin so far wins
- Models should beat these to add value