Draft Theory: On Risk and Reward

Hockey is full of time honored cliches: “We’re going to take it one game at a time” “A two-goal lead is the hardest one to hold on to in hockey” “When you put the puck on net, good things happen” “We need to play a full 60 minutes” “We need to give 110%” “Your goalie has to be your best penalty killer”. I’m not sure if it’s that we love cliches, or the play-by-play guys and players love to repeat them.

In any case, one of the most overused cliche’s as it relates to prospects is “He’s a high risk, high reward player”, but in many cases this term is pretty inaccurate. 

I’ll explore the relationship between risk and reward when evaluating prospects after the jump.

On a recent post by Garret Hohl, of Jets Nation and Hockey Graphs, he reminded me of an excellent graph from Michael Schuckers in his study of the value of an NHL draft pick: 

Here is his summary of the probability of a prospect playing over 200 NHL games by position and by draft round: 

As we can see, after the early first round picks, the probability of drafting an actual NHLer plummets.  

By definition, other than the Connor McDavids and Jack Eichels of the world, virtually every prospect is a high risk pick, so unless you’re talking about an elite sure thing, of course the a prospect carries a high amount of risk. 

So when we’re looking at a particular prospect, it’s not their inherent risk itself that is important, it is their risk as compared to the inherent, or average risk, of that draft position.  

Last month, I presented on the topic of prospect evaluation at the DC Hockey Analytics conference (DCHAC). The basic premise of the approach I’ve developed along with Josh Weissbock is pretty simple: if we can find a list of players (ie: a cohort) who most closely match the attributes of a particular prospect, we can use this information to help predict whether or not the prospect will be successful or not. It’s a long read, but to summarize the results, in simulated re-drafts, teams that utilized cohort success rates along with NHL equivalency adjusted points/game scored 23% more points than the players actually drafted by the NHL teams. This was despite the test method selecting players actually drafted by NHL teams only 58% of the time, indicating that 42% of the players tested by our model never really had a chance at NHL success without teams invested in their development. 

Since this data was presented last month, we’ve refined our model considerably, and now our database returns results based on the closest comparables for a prospect using the following criteria: 

– League 

– Age (matched to actual birth date, not just birth year)

– Point production 

– Height

The output from our database is a list of comparable players, Prospect Cohort Success %, or PCS%, which is the number is the percentage of comparable players who played over 200 NHL games (Jonathan Willis explains why 200 games is important here), and PCS Pts/GM, which is the NHL points per game of the players within the cohort of comparable players. 

To give it a test drive, I queried the top undrafted players from the 2014 draft in terms of PCS%. Here were the results: 

The list above, is pretty interesting: 

– Tkachyov (or Vladimir Tkachev as you probably know him), of course, is the undersized forward who wowed the Oilers brass in a camp invite, and eventually signed a contract with the Oilers, only to have it later ruled as ineligible. 

– Sebastian Aho (the Swedish one) seems to be a consensus 2nd/3rd round pick in his second year of draft eligibility. Corey Pronman has him ranked as the 42nd best prospect in this draft class, and CSS has him ranked as the 13th highest international player.

– Joe Hicketts was signed by Detroit after the draft, and went on to beat out many highly touted defensemen to play for Team Canada at last year’s World Junior tournament, winning a gold medal along the way.

– Rihards Bukarts is a key component of the Brandon team which just lost to the Kelowna Rockets in the WHL finals. More importantly, he’s a Latvian with a pluralized first and last name, and I just can’t get enough of them right now. 

All of these players had higher than average probabilities of success than the 7th round averages per the Schuckers study. In essence, these players had lower than average risk profiles, relative to players typically selected in later rounds.

In addition, each prospect had PCS Pts/GM rates that should have piqued the interest of NHL scouts. Some of these players, like Sebastian Aho, would have been an excellent pick in earlier rounds as well. Given his size he would most definitely been mis-characterized as a “high risk, high reward” pick. In fact, given his PCS% and PCS Pts/GM rates, Aho could have been accurately described described as a “low risk, high reward” pick relative to the other prospects on the board in the third round. 

This brings us to the case of the two most recently signed Canucks prospects: Jordan Subban and Mackenze Stewart. Here is the data from their draft years: 

What may come as a surprise to many, both Subban and Stewart’s PCS% actually exceeded the probability of success of their draft position, although Subban much more so than Stewart. The following graph show’s Stewart and Subban’s PCS% relative to the probability of success of their draft position: 

As we can see, neither pick was particularly risky, relative to the draft position, although both players would correctly be considered to be long shots to make the NHL. Subban’s PCS% is lower than one may expect, but the reality is that there really hasn’t been many players his size who have made it. More importantly, it was significantly LESS risky than the defense picks in that round, historically. 

In terms of Stewart, he unquestionably has NHL size, but players like Stewart who haven’t been able to demonstrate offensive aptitude haven’t fared particularly well historically, so we see his PCS% only slightly higher than the historical rate of 7th round defense draft picks. 

The real difference between these two is the reward. The graph below shows the Pts/GM of 2014-15 NHL Defensemen as well as the PCS Pts/GM of Subban and Stewart: 

Subban’s PCS Pts/GM was 0.24, while last year only 60 NHL defensemen produced at at that rate or better. If Subban can beat the odds and becomes an NHLer who produces consistent with the rate of his peer group, he’ll be a very valuable player to the Canucks in the future. On the other hand, there were 170 NHL defensemen who produced at a rate of 0.135 pts/gm or better, meaning that if Stewart bucks the odds and finds his way to the NHL and produces consistent with his peer group, he’ll roughly have the value of a replacement level defensemen, at least offensively.  

Through the use of PCS% and PCS Pts/GM we can estimate both the risk, relative to draft position, and reward, relative to the player’s closest peers who have come before them. By definition, replacement level players are easy to find through free agency or the waiver wire, so it really makes little sense to use valuable assets on players with replacement level upside. 

The data presented at DCHAC indicates that when teams take these quantitative factors into consideration, there is the potential to improve their drafting significantly. Inherently, every draft prospect carries a high level of risk – that’s a given. However, as we’ve seen from the players passed over in 2014, teams consistently pass over players who carry lower risk, and higher reward profiles. 

And with that in mind, the focus of future posts will be players worth targeting in the 2015 draft. Stay tuned.