Game Star Ratings

I’ve added a star rating to each game. It measures the “enjoyability” of a game based on a few different factors. There are many elements to a game that can make it enjoyable to the unbiased fan. I’ve tried to include the most important of these.

The rating system ranges from 0 stars to 5 stars and goes in increments of .25 stars. The average game will be around 2.5 stars.

Leverage Index (aLI)
The first and most important element is leverage index, which measures the importance of each situation in the game. The more crucial a moment in a game is to the outcome, the higher the leverage index will be. A leverage index of “1” is average. A leverage index of “3” is equal to three times as important as the average play. FanGraphs has a primer on LI for those interested.

In the game rating formula, I use average leverage index over the course of the entire game. I could have chosen to just go with the top X plays in a game, or the number of plays over a certain threshold, but I felt the average over the course of the game is best suited to gauge the intensity of the entire game.

Win Expectancy Change (WE+/-)
Next is change in win expectancy per play. Suppose an RBI single increases a team’s win expectancy from 55% to 65%. That would obviously be an increase of 10 percentage points. I calculate the average absolute value of WE change over the course of the entire game and use that number for my rating formula.

The average play in the average game will have a win expectancy change of about 3.3 percentage points. Bigger and more exciting plays will increase this number, while plays in blowout games will do the opposite.

Leverage index and win expectancy change very likely have a high correlation, which would cause these elements to be “double counted”. I have taken that into consideration and am fine with it since they are the most important factors in gauging a game’s intensity.

Championship Leverage Index (CLI)
Championship leverage index is similar to in-game leverage index (above), in that it gauges the importance of a single game as opposed to a single play. The game importance is measured in how much a team’s probability of winning the World Series changes in a win versus a loss.

The average game will have a CLI of 1 and is equal to the average game on opening day. In the 2nd Wild Card era (2012-present), the average game on opening day can change a team’s chances of winning the World Series by 0.59 percentage points.

The CLI used in this ratings formula is the average of the two team’s CLI for this game.

Examples: A team that is already eliminated has a 0% chance of winning the World Series. A win will not increase their chances, so their CLI will be 0. The same goes for a team that has already clinched their division. A division title ensures that a team is 1 of the 8 teams in the postseason tournament, meaning they have a 12.5% chance of winning the World Series. A win or a loss after clinching the division will not change this number. But a one-game playoff for the division (game 163) is a “win or go home” scenario and will have a CLI of around 21, since it is 21 times more important than the average game on opening day.

Comeback (CB)
The final element is comeback, which is defined as the highest win expectancy the losing team reached during the game. A comeback can range anywhere from 50 percentage points to 100 percentage points. A comeback of 100 percentage points means that the losing team had a 100% chance of winning, but still managed to lose the game. A comeback of 50 percentage points means the losing team was never able to increase their win expectancy above the 50% level at the beginning of the game and likely means the game was never much in doubt.

Formula and Weights
Each of the four elements (LI, WE+/-, CLI, CB) are individually compared to a large sample of games ranked in a percentile. These percentiles are then weighted and combined to create the star rating. The weights are:
aLI = 1.5
WE+/- = 1.5
CLI = 1
CB = 1

Example: A game has an average leverage index of 1.25, an average win expectancy change of 4.5 percentage points, a championship leverage index of 1.55, and a 85% comeback. Their percentiles and weights are:
aLI = 70 * 1.5
WE+/- = 82 * 1.5
CLI = 92 * 1
CB = 90 * 1

Their sum is 410. This number is divided by 25 and rounded to the nearest whole number. It is finally divided by 4 to give you the star rating. This game would be a 4 star game (410 / 25 = 16.4 = 16 / 4 = 4).

Elements of a game not currently included in star rating system
Individual game performances and milestones. A player hitting 4 HR in a game is exciting and uncommon and makes each of the at bats more important. A pitcher taking a no-hitter or perfect game late into the game has the same effect. These types of elements are currently not included, but are “on the table” for future versions.

Star Players
One could argue that the more superstar players in a game could make it more enjoyable. This rating system does not take the players superstar status or skill level into account.

Special Games
While Derek Jeter’s final home game was exciting in its own right, I would argue that it was even more enjoyable since it was his final game at Yankee Stadium. This rating system doesn’t take these rare situations into account.

The Home Crowd’s Enjoyment
As mentioned above, this star rating measures the enjoyment for the unbiased fan. The home crowd may have a different definition of an enjoyable game based on whether their team wins, but this system makes no such distinction.

This entry was posted in Announcements, General, Statistical Analysis. Bookmark the permalink.

7 Responses to Game Star Ratings

  1. Mike G says:

    Wow! Amazing! I was looking for something just like this to pick the best of the games from the day before to watch on MLB.tv.
    If you’re looking to add to it, my top suggestion is to give weight to the starting pitchers, since that varies more than anything else on a team from day to day. Maybe use the Established Level you’ve calculated.
    But really, fantastic job with this and the site as a whole!

    • Dan Hirsch says:

      Thanks Mike! I’m still on the fence about this. One part of me says that a game can be exciting no matter who is playing, but I also understand that people would rather watch Kershaw, Harper, Trout, etc, rather than replacement level players. I think I’m going to leave the current formula as-is and maybe revisit it in the future.

  2. Simon says:

    Dan — great addition to the website! For the purpose of sifting through MLB.TV games, I have made similar rankings for myself in previous years. Average leverage index is a great ranking, and I will also add in variables such as the quality of the two teams, the time of the game (shorter = better), and the attendance (bigger crowd = more atmosphere and generally a more important game).

    This year I have taken a different approach, and ranked the games based on who actually played in them (inspired by this article: http://fivethirtyeight.com/features/the-best-mlb-all-star-teams-ever/). The formula is just the sum of ((plate appearances or batters faced) x WAR) for both teams, divided by total plate appearances for both teams in the game. Based on this measure, the best game of the 2015 season was the Cubs/Pirates one-game playoff on October 7 (Jake Arrieta vs Gerrit Cole) — 67 plate appearances with a weighted average WAR of 7.95 (I believe I used Fangraphs’ FIP-based WAR for the calculation).

    The benefit of this approach is that the highest-rated games feature the best pitchers going deep into the game, and the star players are in the lineup (rather than taking a day off). The drawback is that the starting pitcher faces ~25 batters per game, and has a much bigger impact on the game’s weighted average WAR than an individual position player.

    Here’s the top 10 for 2016 so far (using Fangraphs’ FIP-based WAR, which explains why the Marlins/Mets game tops the list — Jose Fernandez has a 4.37 ERA but a 2.42 FIP). The list is dominated by Noah Syndergaard and the Cubs.

    Rk. Date: Wtd WAR (Score: gameday blurb)
    1. April 12: 0.56 (Marlins 2 at Mets 1: Martin Prado recorded the go-ahead pinch-hit sacrifice fly in the eighth as the Marlins have beaten the Mets twice in a row. Noah Syndergaard struck out 12 batters in seven innings, while Jose Fernandez settled down after a shaky start to retire 10 straight through five frames.)

    2. April 18: 0.56 (Mets 5 at Phillies 2: David Wright hit his first and second homers of the year to back another dominant Noah Syndergaard start — eight strikeouts over seven innings — in the Mets’ win over the Phillies at Citizens Bank Park. Philadelphia starter Jerad Eickhoff was sharp as well, allowing two runs on five hits and while striking out nine over seven innings. Lucas Duda and Neil Walker hit back-to-back solo shots in the eighth.)

    3. April 19: 0.53 (Cubs 2 at Cardinals 1: Jason Hammel was sharp over six innings and paced the offense with a two-run knock as the Cubs took a 2-0 lead in the rivalry series. Yadier Molina’s booming RBI triple off the left-field wall was the only damage against Hammel and the Cubs, who also turned an inning-ending double play at the plate on an amazing throw from Jason Heyward.)

    4. April 8: 0.51 (Cubs 2 at Diamondbacks 3: D-backs slugger Paul Goldschmidt tied things up in the eighth and then Yasmany Tomas singled home pinch-runner Chris Owings with the game-winning run in the ninth to hand the Cubs their first loss. Chicago threatened to break things open in the fifth when it had runners on the corners with no outs, but after Ben Zobrist struck out, Arizona turned a 3-2-5-2-6-9 double play.)

    5. April 15: 0.49 (White Sox 1 at Rays 0: Chris Sale allowed only two hits and struck out nine en route to a shutout in the White Sox 1-0 win over the Rays. Rays starter Jake Odorizzi was sharp as well, striking out six over seven scoreless innings. Melky Cabrera plated the only run of the game on a single in the top of the ninth off Alex Colome. The win was the White Sox fifth in a row.)

    6. April 25: 0.49 (Reds 3 at Mets 5: In the seventh, after Michael Conforto and Lucas Duda backed Noah Syndergaard with deep flies, Mets newcomer Neil Walker hit the club’s third tiebreaking homer of the night, his eighth of the season, to finish the Reds. Joey Votto’s tying hit in the seventh was spoiled.)

    7. April 17: 0.44 (Giants 1 at Dodgers 3: Kenta Maeda limited the Giants to one run on four hits and fanned seven to outduel Jeff Samardzija, who took a hard-luck loss after giving up three runs in six-plus innings. Joc Pederson’s two-run homer proved the difference as the Dodgers won the three-game series to take over first place in the NL West.)

    8. April 15: 0.44 (Giants 3 at Dodgers 7: Enrique Hernandez homered twice and doubled for four RBIs off Madison Bumgarner and Clayton Kershaw held the Giants to two earned runs over seven frames to take the opener of the three-game clash between NL West rivals. San Francisco’s defense made it tough on its ace, committing three errors.)

    9. April 5: 0.44 (Mets 2 at Royals 0: Noah Syndergaard was in no mood to celebrate after the Royals received their World Series rings, striking out nine over six innings and clamping down whenever K.C. threatened. Neil Walker’s homer, the Mets’ first of the season, was the difference.)

    10. April 9: 0.43 (Cubs 4 at Diamondbacks 2: D-backs ace Zack Greinke was outpitched by the Cubs’ Kyle Hendricks, who gave p four hits in 6 2/3 innings, fanning five. The offense gave him plenty of support, particularly Ben Zobrist, who went 3-for-4 with a double, two singles, an RBI, a walk and a run scored. Greinke struggled with command, giving up four runs in six innings, including three in the first frame, but he did strike out eight.)

    • Dan Hirsch says:

      Interesting. I assume that this means you have to recalculate the game score each day since the player’s WAR totals will change. I kind of like having the game rated right away without having it change later in the season. I think I’d prefer to use a “true talent level” on the day of the game to estimate the quality of players in the game. Something like Established Level, as Mike G mentioned.

      As for time of game, I’m not sure I agree that shorter is always better. I’ve personally enjoyed many long and short games, but I don’t think it is the time of game that makes it enjoyable. Sure, if a game is 4 hours long and never close, I will agree that it isn’t enjoyable But that is already dealt with in the current method.

  3. Simon says:

    I would’ve preferred to use something like projected WAR, but Fangraphs has a downloadable .csv file of season-to-date WAR, so I took the easy route. As the season progresses, the projected vs season-to-date debate will become less important. Also the season-to-date data captures the current-year storylines — I like checking on the Trevor Storys of the world who are off to pretty good starts but may not have high projected WARs. I’m not 100% happy it is the right approach, but it working well enough.

    The rankings do move around a bit from game to game. After good outings last night from David Price and Johnny Cueto, all previous games involving them have moved up in the rankings. Price’s start against the Blue Jays on April 16 has ticked up to 7th on the list. I like though that as players perform well, overlooked games from earlier in the season start to rise to the surface. One of the best-ranked games from last year was on April 6 — Corey Kluber vs Dallas Keuchel. It takes on more importance in retrospect when Keuchel emerges with a Cy Young season.

    One of the problems I have with ranking games based on metrics like average leverage index is that those metrics tend to highlight certain ‘types’ of games. Games with high average leverage index tend to be either extra-inning games or low-scoring pitchers’ duels. Games with high ‘comeback’ ratings tend to be high-scoring games that drag on for 4 hours. Games with a high championship leverage index tend to feature two 83-79 teams fighting it out for the last wild card spot. After a while you get bored of watching these types of games. I’d rather watch two 97-65 teams go at it, with their best pitchers on the mound, and I don’t mind if the score is 5-1 (low average leverage index) and both teams are leading their divisions by 10 games (low championship leverage index).

    I think you get around these issues by combining several different metrics, although of the top 9 games on your list for this season, 7 of the 9 went extra innings, and the two 9-inning games had scores of 9-7 and 12-10.

    I’m very happy you’re running this list every day, and I look forward to watching some of these games. The links to the video highlights and baseball reference pages are a nice touch as well.

  4. Daniel says:

    If you want to watch exciting baseball games without already knowing the score/outcome/highlights, I’ve been blogging at http://baseballrewatch.com/ and listing the best 2-5 games from each day of the season. I use a combination of LI, WE change, pitcher’s game scores, and mix in a few subjective elements like highlight reel plays and newsworthy events.

    I launched the site early 2017 and listed games from 2016 if you want to catch something older, and have also been listing each day’s best games the day after, plus weekly and monthly bests. Hope you can enjoy!

Leave a Reply

Your email address will not be published. Required fields are marked *


8 − three =

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>