Friday, November 20, 2015

Week 12 College Football Math

I apologize for the late post. I've been running around like a crazy man this week.

It's an elimination week in the Big-12. Baylor, TCU, and Oklahoma all need to win to keep their CFP hopes alive. Oklahoma State, being undefeated, would still have some CFP hopes if they lost, but it would be much much better if they won.

I wrote and published a long post on some of the model's inner workings, go check it out.

Tuesday, November 17, 2015

Dashboards updated through week 11 & CFP

Through week 9

Last week, the CFP Committee Model predicted the rankings would be:
  1. Clemson
  2. Alabama
  3. Ohio State
  4. Baylor
  5. Notre Dame
When the rankings came out they were:
  1. Clemson
  2. Alabama
  3. Ohio State
  4. Notre Dame
  5. Iowa (?)
  6. Baylor
I post that not to brag, but as a lead-in to this week's rankings. The model swapped Clemson and Alabama. The AP poll or the coaches poll would never rank a 1 loss team #1 with so many undefeateds remaining. But the CFP committee? They might; they're a creative bunch. I'm interested to see how they handle Oklahoma. On one hand, they crushed Baylor this week but on the other hand they were ranked 12 last week, and it's just one game. I'm not sure how much appetite the committee will have for moving them up.

Saturday, November 14, 2015

Week 11 CFP Math

Are you ready for football?

I wrote a long post (with lots of pictures!) that gets into the single most powerful ideas behind my model, go check it out.

I'm continuing to experiment with new ways to analyze college football data, especially as it relates to the CFP. I have something new to try this week, and there's more on that below. First though, let's look at the graphic of dessssssssssssssstiny.

  • Controls Destiny: All the undefeated power 5 teams control their own destiny, no surprise
  • Almost Controls: Filled with teams who: have one loss, are blue chip programs (also, there's Iowa). They are almost certain to get in if they win out. They are behind 5 undefeated teams right now, but teams lose, shake-ups happen, and if one of these teams wins out they're in
  • Needs Some Help: Teams who have 1 loss, but started the season with lower expectations, and have smaller names. 
  • See you next year: Intuitively, I disagree with the model here. I'm not sure why it's hating on Michigan State and UNC. Either of these teams winning out (and beating Ohio State or Clemson along the way) would have a strong CFP resume. Houston looks about right though - they need to win out and hope for chaos up top.

Week 11 Games - High CFP Leverage

I was curious which games had serious CFP implications, so I sat down to try and figure it out with math. It turned out that measuring "CFP Implications" was trickier than I expected. I started by adding the simulation output that finds, for each team, what their CFP odds would be if they won the game and what their CFP odds would be if they lost the game. For example: in the graphic below OU's CFP odds will be 33% on average if they beat Baylor this weekend, and 1% if they lose.

However, those statistics is not sufficient to measure CFP leverage. For example: Notre Dame is in a strong position to make the CFP. If they lose to Wake Forest this weekend that will change; their CFP odds will crash to near 0. So why isn't Wake Forest @ Notre Dame included in the list below of high leverage CFP games? Because there's almost no chance Wake beats Notre Dame; the game will be an uninteresting blowout. Were I to include only the "if they win" and "if they lose" numbers, I'd be stuck with games like Wake @ Notre Dame at the top of my list. I needed to do better. 

I added a measure for how close the game is likely to be, and got the list below. I'm happy with that result for the time being. Especially since it pulled out OU @ Baylor. The final formula includes: CFP odds if they win, CFP odds if they lose, and likelihood of losing (closer games are better), and it gave us the following list of games.

Duh. I could have picked this game as the top CFP game without any math at all. Oklahoma and Baylor both have a good shot to make the CFP, the loser will see their CFP odds plummet, and the game is expected to be close. 

Alabama looked like the best team in the country last week, they stopped the Leonard Fournette freight train in its tracks. No rest for weary though, as they head to Mississippi State, and they'd better be up for it, because a single loss could doom them.

For years this has been the flagship game in the Pac-12, and it is again this year. In past years Stanford has spoiled the season for Oregon several times; I'm sure Oregon is looking to return the favor and knock Stanford out of the CFP.


If OK State loses to Iowa State they will be out; as it should be for anyone who loses to Iowa State


LSU had a rough week last week, but they're still in the hunt. They only have one loss and could sneak back into the SEC championship conversation with an Alabama loss. But this should be a good game, the hogs are always tough.

Happy football!

Wednesday, November 11, 2015

Week 11 College Football Math

The CFP  rankings have been released, we've already had some MACtion, let's do some math.

  • I wrote a long mathy post that gets into the single most powerful ideas behind my model, go check it out
  • Dashboards are updated
  • There's another week 11 post coming that focuses on CFP math

Week 11 games

Oh man. It begins and ends with Baylor/OK, doesn't it. Each team controls its destiny in the Big-12, and very likely in the whole CFP, and this is Baylor's first real game of 2015. In other spots we see both UW and WSU football games on the Top Ten list; it feels good.

Week 10 Results

I added two columns to this week's outcome chart. Their focus is the new published about idea: Posterior Win Probability

The first column shows how that game's outcome translated into PWP. For example, Louisiana Tech rolled over North Texas. Their 56-13 margin of victory translated to a 99% PWP. Since Louisiana Tech's prior win probability (aka model confidence) was just 97%, we say the model was wrong by 2%.

The model was much, much more wrong on New Mexico State @ Texas State. Texas State was supposed to crush NM State, but instead they went down 31-21, a 10-point loss that translates to a PWP of 23%. It's fun to see when the model is super wrong, but there's another reason I like the PWP addition here.

In past weeks, we've looked at whether the model called the right winner. But predictions that accurately get the winner aren't all equal. You can be right, barely right, or even really right. For example, the model correctly picked both Florida and Temple, but while the Temple prediction was awesome, the Florida game turned out much closer than the model expected.

So not only can we now see right pick vs. wrong pick, we can see how right and how wrong. 

Friday, November 6, 2015

Posterior Win Probability

When Toledo upset then #18 Arkansas (in Arkansas!), it shocked the college football world. The game was supposed to be a little warm-up for Arkansas before their demanding SEC schedule. Arkansas plays in the toughest conference, they’re perennially ranked, and they were expected to mop the floor with Toledo while using the game to prep for their gauntlet of an SEC season. It didn’t happen. Toledo looked awesome and escaped Fayetteville with a shocker.

The game was a huge surprise for my college model as well. (Side note: I need a name for my college football model, I’m taking suggestions). The model expected Toledo to lose by 22, and instead they won by 4, a 26-point error.

On the other hand, Georgia Tech’s victory that week (over Tulane) wasn’t surprising. Georgia Tech was expected to crush Tulane, and they did. They won by an impressive 55 points, scoring 65 points to Tulane’s 10. Looking at points scored, though, the model was actually more wrong about the Georgia Tech win than it was about the Toledo win. The model expected Georgia Tech to win by 28, and they won by 55, a 27-point error.

Errors like these, differences between what the model expects for a game and the actual outcome, are way the model updates a team’s rating from week to week. When a team performs better than the model expects, its rating moves up, and when a team performs worse, its rating moves down. This should make sense, yet you would all judge me as totally nuts if I adjusted Georgia Tech’s rating as much as I adjusted Toledo’s rating based on those week 2 results. You would be right, that’s not how I do it. Instead, I use a concept I call Posterior Win Probability (PWP), and that’s the subject of this post.

Posterior Win Probability uses a combination of statistics and historical betting data to calculate a football outcome that is like margin of victory, but with diminishing value for winning by bigger and bigger margins. Using PWP, winning by four is much better than losing by 22 (Toledo), but winning by 55 is only a little better than winning by 28 (Georgia Tech). This allows the model to adjust team ratings that rely on margin of victory, but in a way that’s heavily weighted toward meaningful outcomes.

Posterior Win Probability is what allows the core part of my college football model to be so effective, yet so simple. Once I calibrate the ratings pre-season, the only additional required input is a team’s result from week to week. From there I can predict all the game outcomes for the whole season and use that to predict standings, bowl games, and CFP selection. PWP has been part of the model for a long time, and I think it’s such a cool idea. I’m thrilled to finally share it.

But first, let’s take a step back.

Historical line data is available for every college football game that’s been played for the last 10ish years. You can see what the line was, whether the team won, as well as (in the data set I found) what a bunch of other predictive models thought the line should be (i.e. Sagarin, Massey – spoiler alert, none of them had better predictive power than the lines).

I used that data to calculate how likely a team was to win a game based on the pre-game betting line. For example, teams that were 0-point favorites (pick’em) won basically half the time, teams that were 7-point favorites tended to win around 70% of the time, teams that were 21-point favorites tend to win around 93% of the time, you get the idea.

The graph below demonstrates this correlation. 

10 years of data seems like a lot of data, but most statisticians would laugh at only 9,262 games. As the “smallness” of the data set shows in this graph, it’s bumpy. This bumpiness won’t do if we’re going to use this outcome going forward. For example teams that are favored by 4 have won 64% of the time, but teams favored by 4.5 have only won 60% of the time. This is just noise at these low volumes of games, and we need to smooth it out. We need to fit a parametric curve to this data, one that makes sense and that we can use going forward. Luckily, I still have the actuarial textbook that taught me how to do maximum likelihood estimation*, so I dusted it off and fit a logistic curve to the data. Now it looks like this:

This “smoothed out” version is driven by a simple formula that’s easy to use and produces logically consistent results. This is what I use right now to translate between win probabilities and point spreads.

My model’s raw output is a team’s probability of winning. I compare the opposing team’s ratings and use a formula to calculate win probability based on how those ratings compare to each other. I like to keep my focus on a team’s probability of winning; I find that to be the interesting output. When I bet on sports, I strongly prefer to bet on teams to win rather than cover the spread. After all, we root for our teams to win. No fan ever says “thank god we only lost by 3” or “darn, we didn’t win by more than 7.5.”

However, it’s important to take the model’s win probability output and translate that into a point spread, since that’s the metric that is most commonly used when discussing upcoming games. For example, we all know that Alabama is favored by 7 over LSU this week, but almost none of us know that Alabama has a 70% chance to win this week.

Point spreads are more intuitive to me, too. Every week I use point spreads, not win probabilities, to compare my model’s predictions to Vegas predictions. As an example, this week my model thinks Washington should be a 1-point underdog vs. Utah, while Vegas has UW as a 2-point favorite. Having both predictions in point spreads, in that common frame of reference, makes the comparison easy.

Translating between win probability and point spread is important, and the logistic function above is how I do that. I take a team’s probability of winning and find what point spread it matches up with on that logistic formula.

To return to Toledo vs. Arkansas, the model gave Toledo a mere 6% chance to win that game. Finding 6% on the graph below, we can see it matches up with 22.6 on the point spread axis.

Looking at the model’s prediction, I can calculate their likelihood to win the game beforehand (it was 6%), and I can translate that into an expected points outcome (losing by 22.6) using the process we just discussed. Comparing this prediction to the outcome, I can say they won, and that they won by 4, but doing that gives me no way to evaluate the error on my “6% chance to win” prediction.

I could improve on “they won” by saying they now have a 100% probability of winning.

That also feels inelegant because doing so assigns Toledo (won by 4) and Georgia Tech (won by 27) the same value.

The answer to both these quandaries is Posterior Win Probability.

Recall that I calculate a team’s expected points outcome (lose by 22.6 for Toledo) by mapping their win probability onto that logistic curve. I can also go the other direction, as in the graph below.

Toledo won by 4. On the logistic curve, winning by 4 matches up with a win probability of 61.8%, so I say Toledo’s Posterior Win Probability is 61.8%. That is the number I use to measure the model’s error on that game, as well as adjust Toledo’s (or anyone’s) team rating. Notice how the logistic curve is steep between -14 and +14 and flattens out the further you get from zero. This means that huge losses and huge wins are nearly plateaus because they don’t dramatically impact the likelihood of a team actually winning. However, when an outcome is close, it’s on the steep part of the curve.

Let’s return to the comparison of those week 2 games:
  • Toledo vs. Arkansas
  • Georgia Tech vs. Tulane

The points-based errors are almost the same, but the percentage-based errors are miles apart. They are as apart from each other as American Football is from European football. This is how it should be. Toledo’s outcome was a huge shocker, while Georgia Tech’s outcome was merely a good team running over a bad team. Posterior Win Probability does an excellent job articulating differences between what is expected and what actually happens.

The very different percentage-based errors in these two games led to very different adjustments to the team’s ratings:

Toledo’s team rating went up by nearly 25%, and Georgia Tech’s rating barely budged. This is the elegance of Posterior Win Probability:  it uses margin of victory but gives diminishing marginal value for additional points scored, to the point that blowing out a cupcake team by 70 instead of 50 means almost nothing, but winning by 21 when you were expected to win by 1 makes a huge difference.

This is exactly how Posterior Win Probability is supposed to work. Huge losses and huge wins fall on the nearly flat part of the logistic curve and aren’t strongly differentiated from one another. Conversely, close losses and close wins are treated with more significance.  

Now that you know everything there is to know about Posterior Win Probability, I hope you’re as enthused about it as I am. PWP is a core part of my college football model, as well as this year’s new and improved CFP model, and I’m excited that I can now reference it in my writing and analysis. I’m also eager to explore additional potential uses. More to come!

*maximum likelihood estimation is a process of estimating the parameter(s) of a given statistical model. You can read way more about it here

Tuesday, November 3, 2015

Dashboards Updated

Through week 9

Week 10 College Football Math

A couple of things before I get started:

  • Today is election day. I know it's not a sexy election day the way 2012 was and 2016 will be, but it is election day and you should go vote. I did some math on a few local ballot measures
  • I've been working on a long post that gets into some details behind my college football model's most powerful idea. It's an idea I really like and I'm super excited to be writing about it. Look for that post later this week
  • Washington is favored over Utah, not sure what to make of that
  • Dashboards still aren't updated but will be tonight

CFP Prediction

This year's model includes an improved module for predicting the CFP committee's votes. Tonight we get the first vote. I plan on watching the committee's explanation for their rankings and continuing to tweak the model as appropriate, but here's its first predicted ranking. I'm not sure what to make of #5. Ohio State, or some of the other placements, but I'm sharing it here and opening myself up for comments/feedback/mockery.

The teams are graphed based on their CFP Predictive Score, which is a combination of power rating, accomplishments, loss penalties, and win bonuses. I've chosen not to show the y-axis because the number value itself isn't meaningful yet. For now, it's simply a way to  establish where teams should be ranked and how far apart they are.

Week 10


Week 9 Results

I'll update this post tonight