1.26 Feature Request - track and post PEAK rating

[shannong]

Why:

All skill rating systems assume that win:loss:draw is based on player skill. The element of luck in CCGs essentially makes each player's rating much more volatile, especially at the 3rd standard deviation. The net result is that unlike in pure skill-based games such as chess, go, shogi, etc., It's very difficult to climb to a high rating and hold onto it because skill is only a partial factor. It also makes one's appearance on a "top 20" or "top 100" chart, etc. less meaningful because luck ensures that your presence on that chart will be very ephemeral. Like the "house odds" at a casino, probability is strongly stacked against even the best players being able to maintain a presence on such charts for any significant length of time. For more detail, refer to post #3 in this thread.

Solution:

Instead of players being ranked within the community based on the volatile "current rating" used for matchmaking, rank them for leaderboard purposes instead by the PEAK rating they have achieved within each new "season" (the period after each new ratings reset). This way, players can compare their epeen based on how far they managed to grind up into the 2nd and 3rd standard deviations instead of worrying about how volatile the matchmaking rating is.

Detailed functional spec:

1. The current rating system remains unchanged for matchmaking. The Challenge screen still displays the current "matchmaker" rating.
2. In the database table that tracks ratings, add a new column called PEAK_RATING
3. In your code that updates the database with rating changes after each match result, add lines that compare the newly-calculated rating against the value of the PEAK_RATING column, and if the newly-calculated rating is larger than the current value, update the PEAK_RATING column with the new value.
4. On the STATUS screen, the field currently labeled RATING should be changed to MATCHMAKER RATING. This field still queries whatever rating column you currently use.
5. On the STATUS screen, a new field called PEAK RATING should be added. This field queries from the new PEAK_RATING column.
6. Add a field to the STATUS screen that displays either the current build number or the current season, so that players who take screenshots to "prove" their PEAK RATING can demonstrate that the PEAK RATING was achieved in build n.nn or season n.
7. On the SE website, expand the top 20 page to show instead the top 100, and change the value used for this result set to query the new PEAK_RATING instead of the current matchmaker rating.
8. Add a search query to the top 100 page that enables any player to search for any other player's in-game name. The search result should display the search target's current matchmaker rating AND their peak rating.

Benefits:

The player community will have a more meaningful and intuitive (and less volatile) measurement of their relative epeen. Players will bitch less about "that terrible rating system". The top 100 list provides a better representation of the "top players" in the current player population. Any player boasting about their rating falsely can easily be double-checked by the community. At the end of a "season", any "hall of fame" snapshot of the Top 100 list actually shows a meaningful cumulative result rather than a less meaningful snapshot of very volatile values at an arbitrary point in time.

[Codepunk]

If we rank people based on their peak rating, it seems no more meaningful to me. Correct me if I'm wrong, but this is the way I see it - say somebody manages to find a brand new strategy that is fairly effective after 1.26 comes out. Say they climb the rankings to 500 while the community attempts to work out a counter strategy. One is found, and the player doesn't actually have the skill to adapt. Thus their rating drops to say, 250, and remains there for the rest of the season.

If they peaked at 500, that's not at all an accurate representation of their skill. That player might remain in the top 100 but not actually deserve to be there. The problem with judging by peak rating is it fails to take into account how far and how fast they may have dropped from that position.

If anything, I think top 100 should be based on something like win/loss ratio, as that seems the best measure of skill to me...the problem with this is, different players will have played different numbers of games, ie someone with one win and no loss would have a 100% win rate. However, I believe that some math that takes the number of games into account could still be worked out.

At the very least, some manner of average rating might be more accurate...but peak rating allows people to get in just by lucky, quick spikes.

[shannong]

If we rank people based on their peak rating, it seems no more meaningful to me. Correct me if I'm wrong, but this is the way I see it - say somebody manages to find a brand new strategy that is fairly effective after 1.26 comes out. Say they climb the rankings to 500 while the community attempts to work out a counter strategy. One is found, and the player doesn't actually have the skill to adapt. Thus their rating drops to say, 250, and remains there for the rest of the season.

If they peaked at 500, that's not at all an accurate representation of their skill. That player might remain in the top 100 but not actually deserve to be there. The problem with judging by peak rating is it fails to take into account how far and how fast they may have dropped from that position.

If anything, I think top 100 should be based on something like win/loss ratio, as that seems the best measure of skill to me...the problem with this is, different players will have played different numbers of games, ie someone with one win and no loss would have a 100% win rate. However, I believe that some math that takes the number of games into account could still be worked out.

At the very least, some manner of average rating might be more accurate...but peak rating allows people to get in just by lucky, quick spikes.

If you really digest the links I gave in the OP, you'll see that it is impossible for even the most truly skilled player to maintain a rating at the top, because the element of luck makes it MORE probable with each new match that they will lose rating and backslide. Again, the probability is strongly stacked against you the more you continue playing after hitting a high rating. Just like the house odds at a casino ensure that most gamblers will lose most of their money if they keep betting.

Also, even a quick lucky spike into the top 100 is quite an achievement in itself in a rating system like this one. As you pass 400 and approach 450 it's like swimming upstream against an ever-stronger current with each single rating point you manage to gain.

Finally, you do realize that the entire rating system itself is based solely on, and completely reflects, each player's relative win-loss ratio, right?

[Codepunk]

If you really digest the links I gave in the OP, you'll see that it is impossible for even the most truly skilled player to maintain a rating at the top, because the element of luck makes it MORE probable with each new match that they will lose rating and backslide. Again, the probability is strongly stacked against you the more you continue playing after hitting a high rating. Just like the house odds at a casino ensure that most gamblers will lose most of their money if they keep betting.

I'm not arguing these points in the slightest. What I'm arguing is that using peak rating instead is no better a representation.