Recently, I have been going through old Baseball America organizational top 10s and looking at the predictability for MLB success of the best tool sections. I saw that fastballs and curveballs were the most predictive for pitchers, and that "Best Athletes" was more predictive than "Fastest Baserunner" or minor league speed score. Here, I am looking at infield defense, using the Baseball America's organization top 10s from 2006-2009 (some are missing because site format changes makes it harder to find them all). I looked at both "Best Infield Defense" and "Best Infield Arm". I then looked at their results in the Majors using Baseball Reference's WAA (wins above average) and the major publicly available defensive metrics. The MLB defense section is (MLB career) DRS + UZR +FRAA divided by 3.
For some reason, since Josh Lansford eventually changed to a pitcher, Baseball Prospectus doesn't carry his FRAA data from the minors. The same thing happened with Matt Bush, Sergio Santos, Jerry Gil, Jairo de la Rosa, and Van Pope.
Just like I tried to do (and failed) with Speed Score in the Baserunning post, I wanted to see if we could find a statistical alternative that was more predictive than Baseball America's lists. This is difficult to do, since there isn't leaderboards for these kind of things, but we used to have RTZ data (Total Zone Fielding Runs Above Average, which was calculated by baseballprojection.com), and we still have FRAA and Range Factor.
So I looked at 2008, went through every full season league (A-ball to AAA), found every 2nd baseman and shortstop that played at least 70 games, put in their respective defensive metrics, and then their MLB numbers (if they made it), both their defensive (as done above) and their WAA. You will notice that some don't have FRAAs, especially a few players that didn't make the Majors, that is because of BP errors (the most common one being they were classified as pitchers, and they didn't have their position player stats).
First, the results of the Baseball America players:
51 of the 80 top infield arm players made the Majors. They had an average WAA of .45 and average MLB defense of .33
57 of the top 80 top defensive infielders made the Majors. They had an average WAA of -.23 and average MLB defense of .71
We would certainly expect the defensive players to be better and more certain of making the Majors than the players with strong arms. Strangely, the latter was true, but the stronger arm players were above average in the Majors, while the defensive players were below average (a lot of this probably has to do with Ryan Braun). Comparing them to the speed players, both the arm and defensive infielders made the Majors at a higher percentage, but the speed/athletic players that did make the Majors were better MLB players.
In the minors, the arm players had an average FRAA of 2.16 that season, and the defensive players had a 3.66 average FRAA for their respective seasons. So just as we would expect, FRAA liked the players that Baseball America liked, and as we would expect, it liked the best defensive infielders more than the best infield arm players.
In the "Best Infield Defense" section, the ones with average to above average FRAAs had an average MLB defense of .76 and WAA of -.45. The ones with below average MiLB FRAAs had an average MLB defense of .58 and WAA of .35. Especially considering the fact that the below average FRAA group had a better success rate at making the Majors, there doesn't appear to be much of a correlation. Even if you just looked at the top 20 MiLB FRAAs for the group, they still had a below average WAA and 4 of them didn't make the Majors.
Of course, there could just be a metric bias. That is, maybe the players are not good in reality (whatever that might mean), they just do what the metrics like. At this point, you either trust the metrics or you don't, and make argues for or against based on method and (honestly) authority of the people that created the metrics. Also, FRAA was used in both the MLB and MiLB rankings, so one could argue that there is a FRAA bias, as it is likely to agree with itself on a player, even across leagues and over a period of time.
In the "Best Infield Arm" section, we see a similar result. The above average FRAAs had roughly the same MLB defense as the below average FRAAs, and the below average FRAAs actually had a better WAA.
Out of the 2008 data players, 99 of the 209 players made the Majors (this is irrelevant for the actual study, just a note on how many made it and sort of a jumping of point).
The players that did make the Majors had an average RTZ of 1.94, Range Factor of 4.44, and FRAA of 1.61.
The players that didn't make the Majors had an average RTZ of 1.51, Range Factor of 4.41, and FRAA of -.95.
These results are somewhat expected, the MLB players were better than the non MLB players, but the RTZ and Range Factor data was not significant. The most significant difference was FRAA. How about players that were actually successful in the Majors?
The 21 players that were average or better in the Majors according to WAA had an average RTZ of 5.67, Range Factor of 4.6, and FRAA of 4.79.
The players that did play in the Majors, yet were below average had an average RTZ of .94, Range Factor of 4.4, and FRAA of .74.
So they were much worse in the minors (in 2008) than the average or better MLB players, but they were also worse than the players who haven't made the Majors! This latter result is the main result that is surprising and probably works against the idea that minor league defensive metrics are correlative to MLB success.
Of course, we have only talked about general success, what about when you just look at MLB defensive metrics? 53 of the player that made the Majors were average or above average defensively, while 46 were below average according to the defensive metrics.
The ones that were below average had a (on average) -.17 RTZ, 4.39 Range Factor, and -.92 FRAA in the minors in 2008. The players that turned out to be above average in the Majors had a 3.77 RTZ, 4.48 Range Factor, and 3.85 FRAA. Again, all the metrics seem to have correlation, with FRAA being the strongest (though RTZ not too far behind here).
Of course, the big thing we wanted to test was that if the defensive metrics were better predictors that Baseball America.
Since our 2008 data is just for one season, I took the top 30 players (there may or may not be one representative per team, but it gives a rough estimate of what the top defensive infielder for each team would be like that year. An alternative list to Baseball America) for each set of data.
Top 30 RTZ players (10 runs ended up being the cutoff): 15 made the Majors with an average MLB defense of .53 and WAA of .42
Top 30 Range Factor players (4.84 being the cutoff): 16 made the Majors with an average MLB defense of .31 and WAA of -.5
Top 30 FRAA players (7.9 being the cutoff): 19 made the Majors with an average MLB defense of .31 and WAA of -.06
Mixed results here obviously, but FRAA was the best at picking which players would make the Majors, while RTZ was the best at predicting which players were the best in the Majors. When comparing to Baseball America, we see that "Best Infield Defense" had the best percentage when it came to predicting which players would make the Majors, with FRAA and "Best Infield Arm" virtually tying for 2nd. The "Best Infield Arm" actually was the best predictor of who succeeded in the Majors, though RTZ wasn't far behind. The worst predictor was Range Factor, though "Best Infield Defense" wasn't too far behind.
Comparing the methods to our sample of 209 starting minor league middle infielders, in which less than half of them made the Majors, all 5 methods were relatively successful in picking which players made the Majors, with RTZ being the worst since only half of the elite RTZ players made the Majors. The best at predicting MLB defense (at least according to the defensive metrics) was "Best Infield Defense". The worst was a tie between FRAA and Range Factor, with "Best Infield Arm" not too far behind. Of course, they all had positive ratings.
So we do see that scouting is important for evaluating minor league baseball, something we already knew, and we see that Baseball America does a good job of it, something else we already knew. However, we also see that the defensive metrics we do have for minor league players do a reasonably good job as well. I don't see a lot of prospect writing including defensive metrics, even though they are publicly available and pretty easy to use. It would seem that it can be a helpful tool, at least when evaluating infield defense.
No comments:
Post a Comment