After looking again at the QuesTec Umpire Information System (UIS) product page, I have some additional comments that I think are even more damning regarding the use of this system as a tool for umpire evaluation.
I barely noticed the first time that I read it, the graphic on the page. It shows a left-handed pitcher pitching to a left-handed batter (I would say that they reversed the image but the catcher is right-handed). They have cross hairs around the ball like the photographer is aiming to kill it, and there is a caption that reads:
Umpire Information System
x = -0.68
y = 60.5
z = 3.43
Result Strike 3
Now I don't want to get too picky about the picture used. It was probably just thrown up on the site. Besides the pitch does look like a high strike on the outside corner of the plate even though the catcher is set up a little further outside and the point of view is over the pitcher's right shoulder making it difficult to be sure. But let's not quibble.
I am more interested in the coordinates that they use and how they seem to determine how the pitch is called. The y coordinate caught my eye first because it is exactly 60' 6" (if I can assume the coordinates are in feet). This must measure the distance from the pitcher's rubber and not the release point of the ball, which would be several feet shorter. The official baseball rules have this to say:
The pitcher's plate... shall be set in the ground... so that the distance between the pitcher's plate and home base (the rear point of home plate) shall be 60 feet, 6 inches.
So if y is measuring the distance from the rubber, it would then be at the back of home plate. Interesting, it makes me wonder if that is the sole point that UIS is concerned with. The reason I ask is that the official rules also has this to say in defining terms:
A STRIKE is a legal pitch when so called by the umpire, which
(b) Is not struck at, if any part of the ball passes through any part of the strike zone;
The STRIKE ZONE is that area over home plate the upper limit of which is a horizontal line at the midpoint between the top of the shoulders and the top of the uniform pants, and the lower level is a line at the hallow beneath the knee cap. The Strike Zone shall be determined from the batter's stance as the batter is prepared to swing at a pitched ball.
OK. So a ball is a strike if it passes through any part of the strike zone, and the strike zone is over home plate. Then only measuring at the back of the plate is inaccurate. As Robert K. Adair demonstrated in his great book, The Physics of Baseball, a 65-mile-per-hour curveball can drop out of the strike zone as it passes over home plate and by the same token a high, 95-mile-per-hour fastball can drop into the strike zone while traveling over home plate. Using one point to measure the location of the ball is inherently inaccurate. Its accuracy would be affected more by certain types of pitchers, and therefore cannot be used blindly as a purely empirical evaluation tool. "Wait a minute," you say. "How do you know that other measurements were not taken and this just happened to be the last one recorded as the pitch passes over the plate?"
Well, let's take a look at the other coordinates. X appears to be the distance left to right from the center of home plate. Here x is negative 0.68 feet, which would be 8.16 inches. This would be approximately on the right-hand corner of home, where the pitch in the photo appears to be. No problems there.
The third coordinate, z, is 3.43 feet, which would apparently mean that the ball was almost 3 1/2 feet high. This too would be in the strike zone apparently. Well maybe it is for the average major-leaguer but would it be for Freddie Patek? The rules use very specific measurements based on the individual batter: the distance between the tops of his shoulders and the top of his pants and the height of the hollow of his kneecap while the batter is crouching. How can the system possibly measure these things while the batter is possibly rotating to make contact with the ball? I highly doubt that it does. I would think that it is using pre-calculated measurements to determine the call. That may work for batters of normal height, but what about Patek or Randy Johnson for that matter? What about a batter who crouches down like Pete Rose or one who stands straight up like Jim Thome? So if it doesn't work for everyone, then it's inherently flawed and again a poor tool to use blindly.
The more that I examine the UIS system, the more I have to side with the umpires. It is a nice tool, but it doesn't necessarily measure anything. They could use it to help in evaluations and to help determine which umpires to promote to the bigs, but beyond that it is too highly flawed