Stats: Why we do what we do

Garret Hohl
March 04 2015 08:58AM

You are not a coach, scout, or general manager. Watch the game. 

The numbers are imperfect and do not cover all aspects of the game. Statistics are not everything. The game isn't played on the spreadsheet. Statistics only tell you what happened. Do you think you are smarter then the coach, general manager, or scouts?

Why does the coach choose player A over player B? Why is player utilization often not evidence driven? Why are players with better results sometimes not used more than some players with worse? Why does hockey management tend to gravitate towards certain type of players?

Why does this conflict exist between the eye-test and statistics?

Let's take a look.

Whether you're using the eye-test or statistical analysis, the end goal is the same. It is about rational individuals using evidence to help optimize future wins. The difference is in how one goes about gathering evidence.

Wins and Outscoring: The Desired Result

The end goal is always wins. While "wins is the only stat that matters" is often used as a Straw Man attack on underlying metrics, it has some truth. Essentially all analysis relates to win optimization.

This desire for wins causes some to idolize those that have been successful before. This is why players are sometimes judged by their number of championships (rings: maybe the fanciest stat of them all). This is why some will use team records with and without the player.

People assume those with the desired results are driving those results.

Hockey is a team game though. The contemporary economics of NHL hockey results in even the best teams employing players that are worse than some on even the worst of teams.

Hockey is also a game of bounces. Past wins are not always indicative of future wins due to the massive influence that variance has on results. 

Outscoring is a byproduct of the same philosophy. The team that outscores its opponents wins. The goal is to have as many players as possible capable of out scoring their competition.

This outlook drives analysis using goal-based methods like traditional plus-minus. (plus-minus has other inherent flaws as a statistic but that is for another day).

Like with wins, outscoring is the desired result, but using it effectively. requires large samples to overcome the huge impact of variance.

The Eye-Test: The Reasons Why Players Succeed

The eye-test takes analysis a step further than using desired results.

Traditional scouting looks at the attributes of winning teams and compares them to players. These attributes are what drive on-ice results. They are the reasons why some players and teams beget desired results more often than others.

When comparing two players, it is not typical for one to be better in almost all attributes. There is only one Sidney Crosby. Typically there are tradeoffs. One could be the better skater while the other is smarter. One may be better positionally in the defensive zone while the other breaks the puck out better.

A scout unconsciously takes qualitative information and quantifies it. They may not realize it, but they are unknowingly ranking attributes relative to each other. They are adding up the qualitative attributes, assuming value in each, combining these values, and then comparing.

Statistics: The Impact a Player has that Drive Wins

This leads us to the new wave in statistics, or the underlying metrics.

The method combines the philosophies of the two previous analysis methods. It looks at what predicts desired results while accounting for a player's or team's control over these results.

While Corsi percentage is not the only underlying metric used, it is the most common. This is for a reason. It predicts goal differentials for players and teams in the short run better than past goal differentials. It serves as a proxy for effectively used possession time, and is a byproduct of players with the opportunity to create scoring chances.

Corsi is a desired result that certain players can sustainably achieve more often than others. It is desired in a different way than Goals though. Goals are the ultimate target, while underlying results are desired to achieve the ultimate target. Corsi can be thought of as representing one of the major inputs a player controls in outscoring an opponent.

Analytics is simply using quantitative results to make more informed and likely better decisions.

The Right Way: Using All Methods

GHOHL-1-FEB25 (1)

Graphic provided by Arden Hill

Both the eye-test and statistics have their flaws. The strengths of one often makes up for the shortcomings of the other.

Statistics are powerful. They can accurately describe how a player or team performed in certain areas. 

They cannot breakdown why a player drives particular results if that area is not directly measurable.

A player may be under performing over a short period of time, but statistics may struggle in telling us why. Is a player performing poorly due to the natural variance inherent hockey? Is it due to the player doing something different? Is the player holding on to the puck too long? Is he not going to the 'dirty' or 'inside' areas of the ice?

Statistics may also struggle in finding the answer. Does the player lack the ability to win board battles and could use a linemate who could assist them in that area? Could the player benefit from attacking the blue line from a different angle? Is the player's confidence causing a rut to sustain itself longer?

The eye-test can tell you the strengths and weaknesses to a player but struggles in quantifying a player's impact in a way that can be accurately compared without bias. No matter what, all people have inherent biases brought on by how one subjectively values certain attributes.

Eyes can tell you that Dustin Byfuglien struggles in the defensive zone. They can tell you that Byfuglien moves the puck well and sustains pressure in the offensive zone. No one though can objectively determine the impact value of each attribute and compare by how much one is superior than the other from simply looking.

Attributes are useless if they do not impact results in one form or the other.

All indications are that coaches, general managers, and scouts are quite effective in terms of the eye-test. The disconnect usually comes in being able to measure the opportunity costs where there is a conflict between different attributes. 

A coach likely knows their fourth line grinder is an inferior player for 5-on-5 and penalty kill situations. The issue is knowing which situation is the optimal deployment. The eye-test cannot compare impact opportunity costs, only attributes relative to each other.

Jonathan Willis once put it best:

In specific instances where eyeballs and analytics disagree, an intelligent man asks why the discrepancy exists and investigates further.

One should not take one method as being right all the time, as both are flawed.

When making decisions and investigating further though, it should be acted out with full understanding that each method has particular strengths and weaknesses.

Closing Thoughts

Contrary to arguments on twitter and in internet comment sections and on message boards, the separation between the eye-test and statistics is heavily overstated.

Skaters who perform better statistically tend to be used more than those who do not. Goaltenders with higher save percentages tend to play more games than those who do not. Draft eligible prospects with higher points per game tend to get drafted earlier than those who do not.

Hockey is not quite the same as baseball when it comes to the "statistical revolution." There does not exist a glut of first-line talents on NHL fourth lines being grotesquely undervalued.

Conflict does exist at times though, which is why the proper and most efficient way of approaching things is to combine the two.

A96da2dbde4a6e338d7c1a0c0287249d
I like hockey. I like numbers. I like the Jets. I have written for the Winnipeg Free Press, Jets Nations, Hockey-Graphs, Arctic Ice Hockey, and Hockey Prospectus. Currently I am managing Jets Nation and you can catch me tweeting @GarretHohl.
Avatar
#1 Jets Man
March 04 2015, 10:23AM
Trash it!
1
trashes
Props
1
props

Well said! I will be forwarding this to all those that I debate hockey with.

Avatar
#2 X
March 04 2015, 12:07PM
Trash it!
0
trashes
Props
1
props

[Corsi] predicts goal differentials for 
players and teams in the short run better
than past goal differentials.
For me, in an article like this, this is the statement that needs to be fleshed out more fully. An explanation of the difference in predictive power of past Corsi as opposed to past goal scoring and the limitations is at the heart of "converting" the skeptical.

Corsi is a desired result..
By your (well, Mr. Hill's) graphic Corsi is in fact an "underlying result" not a "desired result". The desired result would be the actual goals scored.

Your closing thoughts are excellent, what is lost on many who violently oppose any sort of quantitative analysis is that the results of most such analysis is tweaking and incremental improvement, there is no secret Stamkos riding the pine on the 4th line of the NJ Devils to be found. There are a fair number of 4th liners that should be 3rd liners and 1st liners that should be 3rd liners out there and gains to be made by recognizing that these insights are readily available.

Avatar
#3 X
March 04 2015, 12:11PM
Trash it!
0
trashes
Props
1
props

Also, apropos of nothing, I would add that it is my opinion that SportVu is probably more likely to send a lot of teams off in the wrong directions going forward. It will encourage the sorts of micro-analysis that has been popular for a long time and is ultimately unproductive, examining the "clutchness" of players that really just narrows sample sizes to the point of destroying any predictive power a given measure might have.

I also think watching that happen to some of the teams in the NHL will be entertaining.

Avatar
#6 ChinookArchYYC
March 05 2015, 06:28AM
Trash it!
0
trashes
Props
0
props

Really good stuff here and well written. As I gotl to the end of your post Vic Ferarri's words came to mind. "Corsi begets scoring chances, scoring chances beget goals".

Avatar
#7 ChinookArchYYC
March 05 2015, 06:28AM
Trash it!
0
trashes
Props
0
props

Why isn't this cross-post on NHLNumbers.com?

Avatar
#8 X
March 05 2015, 09:16AM
Trash it!
0
trashes
Props
1
props
Garret Hohl wrote:

Re predicting: I've done a lot of this type of work at Hockey-Graphs.com Definitely would suggest there

For the desire comment, it's desired but because you ultimately desire the final product.

It's process vs end results. You desire the end result ultimately, but because of this you desire a good process.

My points were about making this text stronger, I am well aware of your work elsewhere and enjoy it immensely.

My point is that the incongruity of labeling Corsi differentials an underlying result in one place and then referring to it as a desired result elsewhere, while different things are being referred to as desired results as well, undermines the coherence of your article. Simply changing the use of the word "desired" to label positive differentials as an "objective" instead for example would greatly improve the clarity and avoid the awkwardness of writing about "desiring in a different way".

Comments are closed for this article.