Game "Statistics"

[Dae314]

This is a rant.

There are lots of games out there, but most of the competitive ones have some extensive statistics gathering either in game or outside trying to measure a person's skill. Some people argue it's impossible to quantify skill in any game and to just play while others prove this point by shifting their play style (often to their detriment) to maximize the statistics they've chosen to value. Now I'm actually a fan of statistics when used in a natural setting (i.e. where you aren't actively trying to maximize statistics), but I don't use them for bragging or comparing my skill to others. I use the statistics I gather from the games I play to chart progress. I make small changes to the way I play a game and watch my statistics for a while. My goal is always to improve and the trends help me to identify positive play style changes and ditch attributes that are weakening my game.

However there's one thing lacking in basically every statistics gathering program in every game I've seen, and I'd hazard to say in EVERY single information gathering site (rotten tomatoes, myanimelist, metacritic; basically anything that gathers bunches of data for presentation). That is standard deviation. The world seems to be in love with the average. "The average player will score 1000 points in a match." "This movie is about a 7/10 according to user vote." Everyone reports average, nobody reports standard deviation. Why is standard deviation important? Because average is an inherently incomplete and weak statistic. The least information (in my opinion) needed to make a good stab at the true "average" score of something is a mean, standard deviation, and median. Why is the median useful you might ask? Because 50% of the results lie lower than the median and 50% of the results lie higher than the median, and the median is an actual value in the data set unlike average. What if the median score for a movie with an average of 60 were actually 90? That probably means that there's a large group of votes for an obscenely low score, but 50% of the votes ended up over 90. How can you tell that this is so? Standard deviation. With standard deviation you can see how far most of the results are from the mean. If standard deviation is high as we would expect with an average so far off from the median, we've pretty much confirmed that there are lots of low votes and lots of high votes but barely any middling votes (where the mean is).

Bringing things back to skill and gaming, if a player's skill stat (lets just pretend there's a unified skill stat) were of middling performance, but their standard deviation were through the roof, you can accurately say that the player tends to either do very well in a game or crash and burn. Therefore average for a player like that is not a direct indicator of skill, but an indicator of the likelihood of that player having an awesome game vs crashing and burning. This is opposed to an average player with a low standard deviation who would tend to play a steady average game. That kind of differentiation is important. Maybe you WANT the higher standard deviation player on your team. Even if they play like a blind deaf mute one-handed 4 year old in half their games, the rest of the games they play on a higher level than anyone else. Or maybe you already have a performer like that and need someone stable to balance out the swings.

While median is helpful, median is difficult to calculate with large data sets (requires that the data be sorted) so I'll let that one pass, but standard deviation is quite easy to calculate in linear time and it provides a lot of information. Even for things like grades, we rely on averages without any collaborating information. A middling grade with a high standard deviation probably means the student mastered the basics (the beginning) but failed the more advanced. This is in contrast to a student who has a middling grasp of all the concepts basic to advanced (this is of course assuming perfect testing procedure...). Anyway I really wish sites that advertise themselves as "statistics" sites or sites that collect and organize information for display would give just that tiny bit more effort and put a standard deviation by their results. The "statistics" they give would be much more telling if they bothered to do so. Don't even get me started on then calculating and marking significant results...

Edited September 27, 2013 by Dae314