In the first line of his recent Grantland article, Bill Barnwell proclaims, “It's just about impossible to statistically measure the return that each draft pick provides after being selected in the NFL draft.” After giving reasons why this is the case, Barnwell then goes on to…measure the return of each draft pick selected from 1997-2007 (he wanted to look at the first five years of each player’s career). He gives a player one point for starting a game, one-half point for coming in as a substitute, ten points for each Pro Bowl Appearance, and twenty points for Each All-Pro appearance (those players who were both All-Pros and Pro-Bowlers were only given points for their All-Pro appearance). He added up all the of players scores and see how each draft pick from this time period stacks up within his rating system. This would allow him to compare how all players against each other even if they competed at different positions.
Barnwell says acknowledges the flaws in his system, but suggests that it is more of a starting point for discussion. Barnwell is right that there are flaws in his scoring system. The biggest problem is the fact that he is trying to do a relative valuation without actually giving reasons for what a “point” means. For example, what is the reasoning behind giving ten points for Pro Bowlers and twenty points for All-Pros. Are All-Pros really twice as good and/or twice as valuable as Pro Bowlers? It would be very difficult to make any argument that is proves that is the case.
The bigger problem with Barnwell’s approach is that it reflects a trend in the sports industry. Since the publication of Moneyball the book and the release of the Moneyball movie, there has been an explosion in the use of quantitative analysis in sports. This has largely been a good thing as teams like the Oakland A’s, Tampa Bay Rays, and Houston Rockets have been able to effectively compete against teams with more resources. The problem is that now everyone who works in or around the sports industry is trying to find their own version of Moneyball. Yet, finding the right form of quantitative analysis for each situation is very difficult. For example, Bill James (the godfather of sports analytics) published his first book 1977. He is still finding ways to better evaluate baseball from quantitative perspective over 30-years later.
Instead of taking the appropriate approach to this problem, many sports team and leagues take Barnwell’s approach. While it appears to have surface validity, Barnwell’s rating system fails to hold up to even a little scrutiny. It is not just that his rating system is suspect, but the idea that simply starting games makes you a more effective player is questionable. For example, Wins Above Replacement Players statistics in baseball and Player Efficiency Ratings statistics in basketball were developed in part because some players who start are not helping their team win games. In essence, these starting players were less valuable than reserve players and actually could damage a team’s overall performance.
This is not to say that sports organizations should not continue to try to employ quantitative analysis for both on and off the field issues. They should take the right approach to solving the problem. To accomplish this goal, sports organizations need to identify their target metrics and determine what factors help maximize results. In many team sports, the objective of the game is to score more points, goals, etc. than the other team. Therefore, sports organizations should find players who can maximize point differential whether on offense or on defense (or both).
On the business side, one area that has received more attention from a quantitative perspective is sports sponsorship. The primary goal of many corporate partners is to work with teams to grow their revenue streams. Therefore, sports organizations should determine how they can help their partners achieve this goal and track this ability throughout the course of the year. Unfortunately, many sports organizations do not how their corporate partnership inventory drives revenue or delivers a tangible ROI to their sponsors. Block Six Analytics Partnership Scoreboard and Corporate Asset Valuation Model can help sports teams and leagues achieve this goal.
Quantitative analysis is and will be a critical part of the future of sports. Making sure to evaluate the quantitative analysis is something that every sports manager, fan, or sponsor should do to avoid the Moneyball madness.
No comments:
Post a Comment