This past weekend our family was very interested in the NHL Draft, because both my kids have played against, practiced with or know several of the draft-eligible players. What was surprising to me was the number of exceptional players we are familiar with who did not end up getting drafted. Specifically, there were several players who did not get selected that were seen as ‘special’ for several years and almost all of them are committed to play DI college hockey – most of them at top schools.
However, knowing that there were only 225 players selected in the 2024 NHL Draft, I wondered how many of the players selected were American? From there, the math started to kick-in and it helped me realize just how difficult it is to get drafted to the NHL – even in the 6th or 7th round.
Here is how the players selected in the draft breaks out by country:
2024 NHl Draft Picks by Country
Then, when looking at just the American players, you see that by their draft year, almost all the players are playing at the USHL level or higher. Those who aren’t playing USHL or better this past season, surely will by this fall.
2024 NHL American Draft Picks By Level
Odds of Being Drafted
Now let’s estimate how many players were eligible in the U.S. to be drafted. If we look at the number of teams for a single birth year, this year’s draft class was playing 15O hockey during the 2021-22 season. That year there were 113 15O AAA teams according to MyHockeyRankings.To make the math simple, let’s assume most of the Minnesota High School and New England Prep School players also played on a 15O AAA Club team. Therefore there were about 113 x 20 = 2250 players who played AAA or better who were eligible to be drafted this year in the U.S. But only 39 or 1.7% were drafted. Another way to look at it is, that less than 2 full rosters of players ended up being drafted this year. And if you didn’t make the US NTDP team, your chances dropped by ~40% (when you include NTDP alumnni playing elsewhere this past season).
There is Still a Chance
Another interesting piece of data I learned recently is where NHL players were drafted by round. No surprise that the most players drafted were in the 1st and 2nd round. But what was surprising is that there were more undrafted players in the NHL than 3rd round selections.
Bottom line, it is very hard to get drafted. Even harder to make the NHL. But if you didn’t get drafted, there is still hope to make it as an undrafted player.
Each year, USA Hockey hosts a development camp in early August for the top 76 female U18 players – this year for the 2007, 2008, and 2009 birth years (40 Forwards, 28 Defense, and 8 Goalies).
Last month, the initial 63 players were named to the camp. With the final 13 players added at the end of the 16/17s girls camp which takes place a couple of weeks earlier in mid-July. It is my understanding that players from the late-July 15’s girls camp are still not eligible for the U18 camp, regardless of how they performed at the 15’s camp (much to the chagrin of many players and parents). This year, 9 15’s (2009 birth year) players were invited directly to the U18 camp (there were 5 15’s in 2023 and 6 in 2022). So, players not invited as part of the initial U18 list do not have the opportunity to prove themselves worthy at the 15’s camp.
The one key insight from the players invited to the U18 Girls Camp for 2024 is that there were a total of 6 2007 & 2008 birth year players (out of 29 total players from those age groups) who attended the U18 Camp last year, but were not part of the 54 players (from 2007 and 2008) invited directly this year. Instead, those 6 players will need to prove themselves at the 16/17s Camp and earn their invite to the U18s Camp. One way to look at it is, (according to USA Hockey evaluators) there were at least 48 other players from their birth year who showed they were better this year than the 6 players from last year. Another way to look at it is that players should understand that they cannot take anything for granted. No spot is guaranteed.
Note: This is not the first time players have had to start at the 16/17s Camp after participating in the U18 Camp the year before. Last year, at least 2 “downgraded” players did indeed get selected to move on from 16/17s to attend the U18s camp.
This analysis look at the teams selected for the Girls Tier I U16 playoffs. Given the importance of being seen at Nationals from recruiting perspective, I examined the at-large team selections and the seedings.
As a reminder: If you are heading to the Tier 2 girls playoffs, you can still be scouted by DI coaches. This week I spoke with a DI coach who will be in East Lansing and will be looking for the top players from smaller regions that can’t easily play for a Tier I team.
Girls 16U Selections
Below you can see the ranking of the 16 teams who will be playing at the 16U Tier I Nationals. The 3 highlighted teams (Minnesota Magazine, Massachusetts Spitfires and Minnesota Hardware) are the at-large invitations, then there is the host team (Florida Alliance) and the other 12 are the district champions.
The USA Hockey 2024 National Guidebook provides the following description for how they decide on the at-large teams and seedings:
As far as the last team to miss the playoffs, the Mid Fairfield CT Stars had a 95.57 which was 0.93 below the last at-large team, Minnesota Hardware. Similar to the U14s, that is a pretty large difference in ratings (in other age groups I’ve seen a rating difference of only 0.01 or 0.02 between bubble teams) and thus there shouldn’t be much concern about the Stars not being selected based on their rating.
Girls 16U Seedings
For the U16 age groups, the seedings are non-controversial. They match identically to the MyHockeyRankings sorting of the 16 qualified teams.
However, if the selection committee looked at the last 10 games for each of the top 10 teams (as they appear to do in other age groups), there is a reasonable argument to be made for a re-arranging of the Top 8 teams. Specifically, two of the at-large teams could have been seeded higher. The same for Belle Tire. Especially since those teams’ full-year ratings were so close to the teams above them in the rankings.
Want to be scouted at Nationals? Use Champs App Messaging to quickly & easily let coaches know your game schedule
Let NCAA coaches know you’ll be at Nationals and your game schedule. The Champs App Messaging tool is the fast, easy way to send error-free messages to coaches before and during the event.
You can easily select a coach & email template and the message automatically populates the coach’s info, school and your personal information from your Champs App profile. Pick the Upcoming Events template and the Messaging tool with magically insert your upcoming games at Nationals into the message.
This analysis look at the teams selected for the Girls Tier I U19 playoffs. Given the importance of being seen at Nationals from recruiting perspective, I examined the at-large team selections and the seedings.
As a reminder: If you are heading to the Tier 2 girls playoffs, you can still be scouted by DI coaches. This week I spoke with a DI coach who will be in East Lansing and will be looking for the top players from smaller regions that can’t easily play for a Tier I team.
Girls 19U Selections
Below you can see the ranking of the 16 teams who will be playing at the 19U Tier I Nationals. The 4 highlighted teams (East Coast Wizards, NAHA, Team Wisconsin and the Connecticut Polar Bears) are the at-large invitations. Unlike other age groups, there is no U19 team from the Northern Plains district – thus the 4th at-large team. Then there is the host team (Florida Alliance) and the other 11 are the district champions.
The USA Hockey 2024 National Guidebook provides the following description for how they decide on the at-large teams and seedings:
Similar to the the U14 Tier 1 at-large Selections, it is unclear why the Minnesota Empowers and Tradition teams are not at-large teams. Maybe they opted out due to the higher priority of the Minnesota High School hockey playoffs. There is a lack of consistency of Minnesota Elite League teams accepting at-large invitations.
As far as the last team to miss the playoffs, the Pittsburgh Pens Elite has a 95.20 rating which is only 0.24 below the last at-large team, the Connecticut Polar Bears. But it seems this is large enough to make the Polar Bears the at-large selection.
Girls 19U Seedings
For the U19 age groups, the seedings do not match the rankings. Shattuck St Mary’s is ranked #1, but seeded #2 behind Bishop Kearney Selects. East Coast Wizards are ranked higher, but seeded lower than the Boston Jr Eagles. And NAHA is ranked higher than the Mid Fairfield Stars, but seeded lower. As described in the USA Hockey Guide above, it is likely a combination of head-to-head and Last 10 Games that were factors in these seedings.
Specifically, BK Selects beat Shattuck in their only game back in October, and but Shattuck still had a much higher rank in their last 10 games (see below). In addition, S-SM has a 0.65 higher rating than BK, which is quite large. So it seems the head-to-head was the primary factor in the flipping of positions. I suspect this will have impact the motivations of both teams should they meet in the playoff round.
The Boston Jr Eagles won the Massachusetts district, so it seems to make sense that they would be higher than the Boston Jr Eagles.
For NAHA and Mid Fairfield, being seeded #8 vs #9 doesn’t really make a material difference, since they will be in the same division. It just impacts the order of games and who is the home team when they play each other. The two teams never played each other during the season, but Mid Fairfield has the higher rating over the last 10 games.
Want to be scouted at Nationals? Use Champs App Messaging to quickly & easily let coaches know your game schedule
Let NCAA coaches know you’ll be at Nationals and your game schedule. The Champs App Messaging tool is the fast, easy way to send error-free messages to coaches before and during the event.
You can easily select a coach & email template and the message automatically populates the coach’s info, school and your personal information from your Champs App profile. Pick the Upcoming Events template and the Messaging tool with magically insert your upcoming games at Nationals into the message.
Being selected to go to Tier I Nationals is a big deal beyond just competing in the national playoffs to win a championship. Almost every NCAA DI women’s hockey program sends at least one coach to scout players and watch the best teams compete against each other. It gives U.S. players another great opportunity to be seen. While there are many elite players that play for teams that don’t end up qualifying for Nationals – it isn’t the end of the world for them, there are still many other opportunities to be seen (e.g. showcases, USA Hockey camps etc.). However, playing at Nationals is an excellent opportunity and timing to get seen. While the process to make Nationals for District winners and the host team is clear, the at-large selection process is a little murkier. This analysis looks at the at-large selections and how the teams were seeded.
I recently wrote about the selections for the NCAA Women’s Ice Hockey playoffs and how the selections and seedings compared to their MyHockeyRankings ratings. Unlike the NCAA, the USA Hockey National Playoffs actually uses MyHockeyRankings to help select the at-large teams for both youth and girls divisions. On the youth side, in addition to the 12 district winners, there are typically 4 at-large teams selected for Tier I (AAA). On the girl’s side, they also have 12 district winners. However, this is the final year where the girls host team gets an automatic spot in addition to that districts winner. So there are two teams from the Southeast district this year. Next year there will also be 4 at-large teams for Tier I girls.
Candidly, I haven’t taken the time to learn how the Tier II selections are made for Nationals. And I certainly don’t understand how the High School Girls teams are selected – because the last couple of winners have not really played high school hockey teams for their regular schedule, but mostly against other Tier I (AAA) classified teams. So this 3-part analysis will focus solely on the 14U, 16U and 19U Girls Tier I selection and seeds for next week’s USA Hockey 2024 Nationals taking place in Wesley Chapel, Florida.
Note: If you are heading to the Tier 2 girls playoffs, you can still be scouted by DI coaches. This week I spoke with a DI coach who will be in East Lansing and will be looking for the top players from smaller regions that can’t easily play for a Tier I team.
Girls 14U Selections
Below you can see the ranking of the 16 teams who will be playing at the 14U Tier I Nationals. The 3 highlighted teams (Minnestota Walleye, Assabet Valley and Chicago Mission) are the at-large invitations, then there is the host team (Florida Alliance) and the other 12 are the district champions.
The USA Hockey 2024 National Guidebook provides the following description for how they decide on the at-large teams and seedings:
From a selection standpoint, the only team which is not clear is the Minnesota Lakers not being selected for an at-large spot. The two reasons I can think of are a) the first at-large spot already went to a Minnesota district team (Walleye) so maybe the committee didn’t want to take two at-large teams from the same district. The other might be that I have heard Minnesota players care more about their High School playoffs than USA Nationals, therefore getting re-organized (after their High School season ends_ and the cost to play in Florida may not be appealing to some teams. If someone has more information on these decisions, please feel free to provide more information.
As far as the last team to miss the playoffs (excluding the Lakers), the Bay State Breakers were had a 95.48 which was 0.77 below the last at-large team, Chicago Mission. In my experience, that is a pretty large difference in ratings (in other age groups I’ve seen a rating difference of only 0.01 or 0.02 between bubble teams) and thus there shouldn’t be much concern about the Breakers not being selected based on their rating.
Girls 14U Seedings
Pretty much all the seedings make sense with 2 exceptions:
Minnesota Walleye were ranked #1 according to MHR, but Lovell Academy was given the #1 seed. The two teams never played each other during the season, so that could not be a factor. However, when looking at just the last 10 games for each team, it seems Lovell Academy had a higher rating by 0.3 goals – so that may have been the determining factor in giving Lovell Academy the #1 seed.
2. The Minnesota Green Giants and Philadelphia Jr Flyers were ranked #5 and #6 respectively according to MHR. But the Jr Flyers were given the higher seed. But looking closer, both teams had an identical 95.81 rating. And the Jr Flyers had a 0.3 higher rating over their last 10 games – so this could likely be the determining factor for promoting the Jr Flyers.
The next posts will discuss the USA Hockey Nationals Girls U16 and U19 selections and seedings.
Want to be scouted at Nationals? Use Champs App Messaging to quickly & easily let coaches know your game schedule
Let NCAA coaches know you’ll be at Nationals and your game schedule. The Champs App Messaging tool is the fast, easy way to send error-free messages to coaches before and during the event.
You can easily select a coach & email template and the message automatically populates the coach’s info, school and your personal information from your Champs App profile. Pick the Upcoming Events template and the Messaging tool with magically insert your upcoming games at Nationals into the message.
Last week, I re-watched the Canada vs Czechia IIHF U18 Women’s World Championship semi-final game. Czechia won the game 4-2 after losing to Canada in the Group A game 8-1. I noticed that two of the Czechia goals were a direct result of major mistakes by the Canadian defenders. I then saw that the shots on goal were heavily in favor of the Canadians when the score was 3-2 at about 43-11 (ended up being 47-12 for Canada).
This got me thinking, how could Canada lose a game that they so clearly dominated in terms of offensive opportunities?
Two thoughts came to mind:
Czechia did an amazing job learning from their group game with Canada and played a defensive structure which minimized Canada’s high risk chances. Czechia then capitalized on the few opportunities they had to score.
Similar to my observations about the USA Hockey U18 selection process, Canada probably puts a bigger emphasis on fielding a team with offensive D than well-balanced defenders. As a result, a couple of defensive mistakes cost them the game.
This is just my hypothesis, I could be completely wrong. And let me be clear, any player that makes the Canada or U.S. U18 teams are exceptional players. They can all skate well, pass well and shoot well. There is no doubt each of them deserved to be on the team. But maybe, they are too similar in their skill set?
When it comes to roster construction, sometimes you need to include one or two 200-foot players to complement the more offensive players. Specifically, there are usually tradeoffs between a great puck handling D with a hard shot vs. someone who is technically better at 2-on-1s, clearing players in front of the net, playing the penalty kill or defending 1-on-1 zone entries. An analogy would be having 7 Erik Karlssons playing defense for a single team – at some point in important games against good teams there will be times you need the D to keep the puck out of your own net. Once again, this is not to say that those highly skilled U18 players aren’t good defenders, but when they go up against the top 2 or 3 forwards on a national team, they will also need to be technically strong on defense.
Let’s look at some data to support why I can understand how a coaching staff would put together a roster with so much offensive power at the U18 level.
2023 IIHF U18 World Champsionship Shot Totals
Canada outshot their opponent by a 6:1 ratio throughout the tournament. They also had over 40 shots per game. I would suspect they felt the team could outscore all opponents as long they didn’t have any major defensive hiccups.
Of course, you are probably asking – weren’t they planning to play the U.S.? Like all Canadian teams, they probably expected and planned to play the U.S. at some point – likely in the gold medal game (given the new format of A/B group play this year, they wouldn’t play each other in group games). So Canada would still need to be prepared to play a high-powered offensive USA Hockey team.
Then I looked at the shot total for the 2023 Under-18 Series which took place in Lake Place between the Canada and USA. Team Canada swept all 3 games against the U.S. team by a combined score of 15-3. Here were the shot totals:
uSA Canada u18 Summer Series SHot Totals
A couple of possible reasons for Canada to justify having highly offensive defenders… Either the felt they could still outscore Team USA and defend well enough to beat them. Or, maybe their D were never tested enough in the USA-Canada Series to expose some of the technical weaknesses against world-class scorers.
So what?
During my experience attending and analyzing multiple USA Hockey camps/events, I have felt that the players being selected have had their offensive abilities overly weighted in the evaluation process. Now, I am fully onboard with most of the high-end, offensive D being the ones being picked. However, including one or two defenders who can also keep the puck out of your own net at critical points of an important game can be the difference between winning and losing in the medal rounds. Having a little more balance on the blue line could be the difference between winning and losing in big games.
The following is a post I wrote almost four years ago about MyHockeyRankings on an old blog I used to publish. Most of it is still highly relevant after all this time. I have added some additional new thoughts at the end of the post.
I spend a lot of time on the MyHockeyRankings (MHR) website. I don’t use the site because I care about the rankings of my kids’ teams, but I do see the rankings, so I am aware of what they are. However, there are many more valuable reasons to use the site that I find really insightful that I wanted to share. I will also put together my thoughts on what the watchouts and drawbacks are for MyHockeyRanks in a separate post.
To start, let’s remind you of why MyHockeyRankings was created and how it is intended to be used. As stated explicitly on their About Us page, the site was set-up to help with scheduling competitive games between clubs. Using a pretty simple algorithm based on goal differential, a rating is created for each team. The difference in ratings between two teams is their expected goal differential (EGD) between the two teams if they were to play each other (subtracting the lower rating from the higher rating). This methodology is used to normalize quality of opponent and calibrate one team versus another. I won’t go into the statistical analysis of the legitimacy of EGD, but the goal differential is only an expected value, and thus there will be a lot of variation in actual game scores. If the EGD is small, then if the two teams would play each other, then it likely would be a competitive game. If the goal differential is large (for me 4 or 5 goals is a significantly large gap) then it likely would not be a competitive game and it may not make sense for the two teams to play in the first place. This is especially helpful if there is a large tournament and the organizers are trying to group teams into competitive divisions or for leagues to draw the line between A, BB and B levels. There is a lot more detail to how and why, but that is the gist of the ratings. Well, now that each team has a rating, it is only natural to rank them. This is where much of the controversy starts with the MHR site, but we’ll discuss the use of rankings separately.
Schedule, Scores and LiveBarn
I track about 10 different teams at various clubs, levels and age groups and I find it much easier to see the schedule and scores in one single location thanks to MyHockeyRankings than going to each individual team’s website or leagues site to find the schedule and/or score. If I want to dive into a particular game then I might go elsewhere, but being able to see which of those teams has/had games in a particular weekend is very helpful. As a bonus, having the LiveBarn icon next to a game (especially if one of the teams is playing at an away tournament) it lets me know that I could watch the game if I wanted to. This feature was remarkably helpful for watching games for a prep school team that we are considering for my daughter. Without the MyHockeyRankings/LiveBarn partnership, I might never have been able to see several of the school’s games.
2. Scouting and Researching
When heading to a tournament it is highly likely that most of the teams we will play we won’t have ever seen or played before. We have used the MHR ratings to help decide which goalie to start in which games. If on the first day of a tourney we have two games and one team is clearly rated higher than the other on MyHockeyRanking, then the coach usually has some insightful information on who to start in each game. For a playoff game one coach asked me to download the most recent games of the team they will be playing, thanks to MHR I was able to see exactly when and where to find the game on LiveBarn. Finally, as mentioned above, being able to find games from potential prep schools for my daughter and watch their play has allowed me to research potential future schools/teams for her to play on based on the quality of their teams, including the LiveBarn video identification.
3. AA vs. AAA?
I care about my kids’ hockey development not the number of letters they have for their level of play. Since there really is no standard of what is AA vs AAA (which is a topic for another post), MyHockeyRankings also helps to compare AA vs AAA teams in an apples-to-apples manner. For example, I have one of my kids playing on a AA team this season that is rated slightly higher than the AAA team in our area. This is not a surprise, and while having a higher rating doesn’t really matter, there is a big difference between the two teams. The AAA struggles to compete and loses most of their games and their players are usually chasing the puck. Instead, nearly all the AA games my kids team plays are competitive and the players are developing a lot more both on defense and offense, especially when the games are close. While MHR didn’t really play a role in the decision on which team my child would play on, the ratings have provided validation that they are playing at the right level for their development even though it has one less ‘A’.
4. Scheduling Games
As mentioned at the start, this is the original intent of the MyHockeyRankings. Last season our team played about 30 games. Which is fine, but a few more would have been nice.
Since there weren’t any local teams that were available in the Spring, we looked to find a few teams that were 5 hours away for weekend exhibition games against 3 or 4 teams. Well, of course I used MHR to find the teams which would be most comparable to our team to reach out to (even though they were a level lower by letter). Once again, this is exactly the intended use of MHR, to help schedule and ensure competitive games.
5. Triangulation Between Levels
My daughter plays on a youth (boys) team and at some point she will switch to playing with the girls. One of the challenges is for me to compare her current level of play with the boys to the girls of her age. However, thanks to MyHockeyRanking and using their methodology, I can triangulate her team’s ratings to other girls teams. Since the local girls team plays enough games in the boys division, I can determine their equivalent boys rating and see the difference to my daughter’s team. It provides an additional piece of data to inform our decision on the where and when she should play in the future. I have used the same methodology to compare between levels between teams from Squirt to Peewee, from Peewee to Bantam and Bantam to Midget. For example, how does a Peewee AA team compare to a Bantam B team? Thanks to MHR I am able to figure out the answer to this question.
6. How are you trending? Last 10 Game Ratings
Figuring out if your team is trending up or down during the season is a pretty important insight. Almost all teams improve throughout the year, but how is your team improving relative to others. Thanks to MHR you can see if the last 10 games are accretive or dilutive to your rating. Also, with the help of a basic spreadsheet you take any time period and figure it out for yourself. As a data geek, I like the ability to analyze this kind of stuff.
Finally, as I admitted at the start, I am aware of the rankings for my kids’ teams. The ranking usually doesn’t vary too much from the start of the season to the end of the season, so once it has been established, not much point paying too close attention to it. But knowing where the team ranks on a national and state/regional level is good to know as a parent. It basically helps me set realistic expectations for where my kids are in their development and what goals to help them set for the coming year.
These are the benefits I have found from using MyHockeyRankings and when used properly it has provided helpful insights for several important decisions for my kids’ hockey development. However, while I have used site for ‘good’, it is pretty easy to use the site in the wrong way. My next post will discuss the watch-outs and using the site in a manner that goes against its original intent.
2023 Update #1: Another strength of MHR has been having USA Hockey use the rankings for at-large invitations to National playoffs. This way the best teams will still participate even if they didn’t win their district championship.
2023 Update #2: I have also used the Women’s College Hockey ratings as part of the college recruiting process. It’s been helpful to see how each of the conferences really compare to each other in terms of level of play. As an example, it shows how competitive the WCHA conference is, and that finishing 6th in that conference would be first or second in a different conference. You could be playing for a Top 10 team in the country and still not make the NCAA tournament. The NCAA might want to change it’s pairwise calculation (which I don’t really understand) and just use MHR for their at-large tournament selections.
Having led performance feedback to dozens (if not hundreds) of people I’ve managed in business, I recognize it is one of the most challenging interactions to conduct in my career. At the same time, I was taught how to take it seriously and learned many of the best practices to ensure a positive outcome from the process for both parties.
It is pretty clear from the parent meeting at the 16/17 Girls camp (and the letter that accompanied the feedback/rating letter) that USA Hockey wants to make no doubt that they are providing a variety of different levels of feedback for each player at the national camps. The details of this feedback were clearly explained in Part I on this topic.
And it is important to recognize that they really do care about giving feedback – because they have dedicated time and resources to the process. I also wanted to also acknowledge that is takes a non-trivial amount of effort to provide detailed feedback to about 400 players across 4 major camps each summer.
At the same time, I’ve spent a ton of time thinking about this topic trying to figure out why almost everyone I have spoken with is disappointed with the USA Hockey Girls National Camp selection and feedback process. And here is what I came up with…
At the end of the day, the current process does not solve the unmet need of the players – which is to have actionable direction on their highest priority development areas. This is because the robustness of the feedback is not commensurate with the level of commitment and investment the players put into making, preparing and attending the camp.
And my reason for this is the following:
The feedback is too generic. For almost all the players, it’s just too simplistic/superficial without personalized examples and not actionable enough.
Here are my recommendations:
Standardize a More Robust Process – The coaches should go through a training session on how the process works and what the expectations are from the coach on the process, content & delivery. All players should receive player-specific information using a common format, but with player-specific examples in the review. While the coaches should have flexibility to adapt the process to their style, each performance review (in addition to the attribute ratings mentioned in Part I) would require the feedback to include each of the following….
Include Player-Specific Key Statistics (e.g. pass completion rates or turnover rates). Nothing is more powerful than data. Being able to show a player how they compared on key attributes compared to their peers makes things much clearer. This became quite evident to me in my analysis of the 16/17 Camp forwards and defenders.
Support with Player-Specific Video Clips – showing a player exactly what they do well and how/when they make mistakes provides “hard-to-argue” credibility to the stats and the coach’s feedback. This would likely use a video analytics system like Instat/Hudl so each player’s shifts could be coded.
These four recommendations would require a significantly greater amount of time and resources than the current effort being done at the USA Hockey girls camps. There may not be time to aggregate everything during that week. But the feedback session does not need to occur at the camp. It can be done a week or two after the camp via a video-call. What matters most is that the players are getting their needs met as to where to focus and improve as a player. Ideally, there would be someone in leadership who was solely responsible for player development and not directly associated with the selections for the U18 camp or team. I know it can be done, because I have seen first-hand more robust feedback processes on the boys side at both the USA Hockey and junior hockey levels.
Final Thoughts
The best organizations focus relentlessly on their customers. One of the biggest ways to ensure these organizations are meeting the needs of their customers is to ask them for feedback. Specifically their overall satisfaction with a question like “Would you recommend [product/service] to a friend or colleague?” followed by “Why?”. In my few years interacting with USA Hockey both as a coach and a parent, I have never been asked for my feedback on the programs I’ve been been engaged with. In essence, USA Hockey has a monopoly on the national team programs so it is understandable that they may not need to be as customer-centric as an Amazon or an Apple. But, if leadership for USA Hockey female national camps wants to continuously improve their program, just like their players do, it would be great if they solicited their own feedback on areas they can improve as an organization. Who knows…maybe getting the gift of feedback on themselves may translate to improved performance on the ice?
Feel free to send feedback on our posts or Champs App to feedback@champs.app
As the governing body of hockey in the U.S., USA Hockey understands the importance of player feedback. At the USA Hockey 16/17 Girls Camp which took place in Oxford, OH this past June, feedback was highlighted in the parent meeting as a key component of the camp. In Part I of this post about the USA Hockey Girls Camp feedback, I wanted to focus on understanding the three levels of feedback utilized during and after the camp. Part II of this topic will discuss my thoughts on how effective the feedback process has been.
1. On-Ice Feedback – During Practice and Games
Just like with their regular teams, coaches were quite consistent in talking to players individually and in groups during practices to share their thoughts on specific, tactical ways to improve a drill or situation. Same for a player coming to the bench during one of the games after a shift – coaches would lean over to players and give advice on what adjustments could be made to improve a player effectives. These situations are quite comfortable for all the coaches at an event like this since most were DI coaches or previous DI players. As I mentioned in my previous post about player feedback, in-game comments are the easiest for a coach to communicate.
2. One-on-One Feedback with one of the Team Coaches
All teams had two head coaches. On about the fourth day of week-long camp, each player had a 10-15 minute conversation with one of their coaches. It is my understanding that most players were asked to do a self-review in anticipation of the meeting. From talking to several parents, the coach-player conversation was then highly dependent on the coach. Some coaches were well-prepared and had video clips to show players as a way to communicate their feedback, some coaches had simple basic priorities for players to focus on while others relied on the player’s self-evaluation as the primary source of the feedback conversation. Given the variance in feedback methods, I suspect the feedback meeting process was not highly structured by the camp organizers.
3. Letter Grade and Player Development Performance Criteria
About four weeks after the end of the 16/17 Girls Camp, my daughter received by snail mail a form letter which included an evaluation which is supposed to serve as a benchmark for a player’s performance at the camp. This entails a letter grade and a rubric on the “Player Development Performance Criteria”. Here are the details.
At the top of the player evaluation sheet, the players was provided a rating of A, B or C with the following explanation
“A” grade = Excellent – ranks in the top 1/3 of players at camp
“B” grade = Good – ranks in the middle 1/3 of players at camp
“C” grade = Below average – ranks in the bottom 1/3 of players at camp.
The Player Development Performance Criteria had 5 possible selections (from best to worst):
Excellent
Very Good
Good
Fair
Poor
Each skater then had attributes selected within two categories. General and position-specific attributes with a selection in one of those five boxes (“X” for each attribute). Here are those attributes:
General:
Makes Possession Plays (i.e. keep team on offense; limited turnovers)
Angling: pressure to take away time/space; dictate play with body/stick
Stick Positioning
Deception
Quick Transitions
Off-Puck Habits & Puck Support
Scoring Ability
Physicality
Athleticism
200-Ft Player
Skating Ability (north/south; agility; speed)
Defenders:
DZone Execution First
Puck Retrievals
Good First Pass or Exit
Win Race Back to D-Side of Play/Net
Wine Board Battles
Deter Offensive Opportunities
Scan to Make Exit Play; Fast Transition to Breakout
Work Well with D-Partner
Gap Control: (North/South & East/West)
Forwards:
Puck Retrievals & Ability to Stay Off the Wall
Ability to Leave Perimeter and Gain Inside Ice
Owning Space with Puck
Scanning/Awareness of Teammates & Opponents
Use Teammates to Make Plays
Zone Entry: Ability to create depth/layers/lanes
Create & Maintain Offense
I don’t know the process that was used to aggregate the evaluators feedback, but am assuming they collected a populated rubric from all the evaluators for a position and then aggregated the data to take an average of the selections. (I hope they used some online tool to aggregate this all, because there are lots of ways to simplify collecting this information). Then I suppose this compiled data was used as the rating for each player’s Development Performance Criteria. I would then assume the average across all Development Performance Criteria was calculated and the each player was force ranked into one of the three tiers to give the letter rating of A,B or B based on which third they ranked.
Other than the rating and the rubric box selection – no other personalized information was included in the feedback. No short paragraph summary (like you would see in a student report card) from the coach or evaluators to provide additional context was provided.
It is important to note that the ratings are based on the criteria described above. If different criteria were used (which will be discussed in the next post), then a player’s rating might be different if those criteria were closer or further away from the capabilities of a player.
In Part II on this topic I will share my perspective on the good, the bad and the ugly of this feedback process.
Similar to the previous post, rather than engage in a subjective discussion on who was selected, I thought it might be helpful to collect some analytical data and metrics to understand how top players performed at the 16/17 camp and compare them to a couple of the players who weren’t selected.
WHY?
When you don’t select the top 3 point-getters from either Girls 16/17 Camp or the U18 Camp, there are bound to be a lot of folks who wonder what the selection criteria is for making it to the next stage of USA Hockey. I don’t know the answer to that question. But I can analyze the video of each shift for several of the top players picked and not picked to see if there is an obvious difference between the two segments. The purpose of this post is not to say who did or did not deserve to be selected to the U18 Camp. Instead, it is to help provide perspective and context to other players and parents the types of metrics that demonstrate the level of play needed to be selected. And ideally, individual players do their own self-analysis to see how they compare.
HOW?
I watched and coded specific attributes for every shift in all 4 games for every player in this analysis using the USA Hockey TV footage. I collected more metrics than are listed below, but I feel that the attributes shown, provide the right amount and level of data to gain an understanding of the level of play for this position. Note: Sometimes the live stream footage didn’t always focus on the area of the ice where the play was taking place, so it is very likely the odd play may have not been accounted for.
Since I only had the time to watch 5 players – I watched 3 selected forwards plus 2 top players who weren’t selected. Those 3 forwards represented a mix of the forward selections. I am not identifying the names of any players because singling out any individual player is not my objective. For full transparency, in this analysis I do know the parents of one of the players.
SO WHAT?
Do I think the 5 selected were in the Top 10 forwards at the camp, almost certainly. Do I think there are 3-5 other players that could easily have been selected instead – also, almost certainly. There is no algorithm to calculate and rank the top players. I don’t know the selection criteria, so whatever they may be (whether well-structured or not) at the end of the day what matters is results. As stated in the parents meeting, the results of the last two U18 World Championships was not the result USA Hockey wanted – so we will see if the current process yields better results.
THE ANALYSIS
2023 USA Hockey Girls 16-17 Camp Analytics for Forwards Selected to Advance to the U18 Girls Camp
Note: Players 1-3 were selected to go to the U18 Girls Camp – Players 4 & 5 were not selected
Some notes on the tracked attributes:
Takeaways = a one-on-one situation where the player gains control of the puck from directly challenging the other player
Giveaways = full change of possession to the other team (e.g. a missed pass, dump in/out, rim or redirected puck)
OZone entries = skating across the blue line with full possession of the puck
Team Shots For/Against do not include shot attempts that did not reach the net. Only SOGs were included.
I am not including the point stats or PIMs for any player since they can already be found on the USA Hockey website
There were additional attributes I tracked like “faceoffs won” but they indirectly show up in other higher-order key metrics. Since not all the forwards played center, I didn’t include the faceoff attribute. But I did want to note, that one player was very good at faceoffs while another was not. The one that won most of their faceoffs did see that reflected in other measurement areas since many faceoff wins led to greater possession time.
OTHER THOUGHTS
From all the players and games I’ve watched, it seems (and it’s only natural) that really good plays are rewarded disproportionately more than their equivalent poor plays are punished (e.g. creating a “wow” scoring chance vs. causing a “wow” scoring chance for the other team). Forwards tend not to surrender many negative scoring chances unless they are somewhat negligent defensively. So, it seems likely that creating offense is highly disproportionately weighted in player evaluation.
Not all players gave the same defensive effort throughout a game, whether it is being tired or laziness. But over the course of four games, it was pretty clear who consistently tried to play a 200-foot game (vs. cheating a little defensively or taking some shortcuts).
Scouting and evaluating is not an exact science. In my humble opinion, most of the scouts/coaches don’t watch any player enough to really get the full picture. It is sampling data – and while it is directionally correct, when there are many players within a close band it is hard to discern who is absolutely the “best” player. And who you pick may vary when you are building a team for a short tournament and need different types of players.
After watching over 20 hours of individual game footage, this process is somewhat exhausting. It takes a lot of work to watch and tag each type of play. I can’t imagine being a scout and trying to watch 10 skaters live on the ice throughout an entire game. At the same time, the insights are quite valuable. I hope that college scouts leverage Instat to watch players individual shifts (if a club/prep team uses Instat) to evaluate the full body of their work rather than just sampling one or two periods of a game during a tournament or showcase weekend. To me, it is hard to watch multiple players in a game rather than on just one player at a time.
Note: We are still waiting to on the written feedback and letter rating that we were told all players would receive. If you are a player or parent from 16/17 Camp who has a received this feedback, please reach out and let me know. Update: We did receive the USA Hockey Feedback on July 27th – I will be writing up my thoughts on the feedback process in a upcoming post.
