This repository contains code and data to accompany FiveThirtyEight's NFL Forecasting game. Specifically, it has:
data/nfl_games.csv
, with FiveThirtyEight's Elo win probabilities for each game.Our goal in providing this repository is for people to be able to figure out how FiveThirtyEight's NFL Elo model and NFL forecasting game work and to provide a loose framework for evaluating forecasts against historical data. This repository does not include assistance in building a predictive model.
eval.py
is the only runnable script, and does the following:
elo_prob1
field, which is the probability that team1
will win the game according to the Elo model.my_prob1
field for every game using code in forecast.py
. By default, these are filled in using the exact same Elo model.my_prob1
against the ones in elo_prob1
, and shows how those forecasts would have done in our game for every season since 1920.Jump in by running python eval.py
. You should see the following output:
On average, your forecasts would have gotten 648.24 points per season. Elo got 648.24 points per season.
This makes sense — right now it's just running FiveThirtyEight's Elo model against itself, so it gets the same number of points for every game.
Open up forecast.py
, change the HFA
(home-field advantage) parameter to 100, and rerun python eval.py
. You should see:
On average, your forecasts would have gotten 603.42 points per season. Elo got 645.9 points per season.
OK, looks like changing home-field advantage from 65 to 100 points isn't a good idea. With that tweak, our generated probabilities perform worse historically than the official FiveThirtyEight Elo probabilities.
Inside the Util.read_games
function, there are three lines you can uncomment to download the 2021 schedule and results to data/nfl_games_2021.csv
. If you run python eval.py
after uncommenting them, you'll see something like the following in the output:
Forecasts for upcoming games:
2021-09-09 TB vs. DAL 82% (Elo) 86% (You)
2021-09-12 IND vs. SEA 59% (Elo) 64% (You)
2021-09-12 BUF vs. PIT 73% (Elo) 77% (You)
The scripts are now maintaining Elo ratings through the 2021 season, and printing forecasts (both from elo_prob1
and from my_prob1
) for upcoming games. Note that our model is more confident in the home team in every game because we've adjusted the HFA
parameter to 100.
Have at it! Some ideas for further exploration: