ZeusWPI / aichallenge

A bot writing contest
https://zeus.ugent.be/bottlebats
9 stars 6 forks source link

Multiplayer ELO #30

Open Procrat opened 8 years ago

Procrat commented 8 years ago

The ELO rating system doesn't properly handle matches with more than two players. So far, we haven't organized any multiplayer matches, so it's not a big deal at the moment.

At the moment, the calculation for a change in rating for some player A in a match against {Pi} happens as follows. The mean is taken over all would-be rating changes if A battled against each of {Pi} separately. This scheme has already been put to good use in other places, but it is made for cases where each match results in a ranking instead of -- in our case -- a single winner. E.g. a loser might have won mid-game against an opponent -- this would be hard to track -- but the rating system would account for a loss against this opponent.

Procrat commented 8 years ago

We could model it as if each participant won/lost against the opponent with the highest rating. I think this would be have realistic outcomes.

wschella commented 8 years ago

This certainly a very pragmatic solution. But I feel it is a bit unfair towards the winner. It is as if he only won against 1 opponent, when he actually defeated many. This system will also result in an overal decrease in elo, since there the lost amount is about n-1 times the won amount(with evenly matched players).

iasoon commented 8 years ago

What about just expanding the formula used for the two-player rating? A's win chance is Qa/(Qa+Qb). Wouldn't that make A's win chance against two opponents Qa/(Qa+Qb+Qc)? That would make the win chance against two equal opponents 1/3 (instead of 1/2 in the current system). When a player rated 1200 battles a player rated 1000, his win chance is about .76 . If he would battle two players rated 1000, his win chance would be .61, and their win chance would be .19. This seems quite okay to me.

wschella commented 8 years ago

The problem with this is that it doesn't scale very well, following example:

Q0: 1400 Q1: 900 . . Q10: 900

Q0 win chance: 1400/(90010) = 0.15 Q1..10 win chance: 900/(1400 + 900 \ 9) = 0.09

This doesn't seem very realistic to me. An increasing amount of bad player should not impact the best players win chance so heavily. If you let 10 cyclists race, with 1 being extremely good, his win chances remain high.

iasoon commented 8 years ago

Right, I didn't try more than three players.

Well, elo rating is specifically about pairwise comparisons. I guess the options are to either not rank with >2 players, or designing your own ranking system.

The more I think about it, multiplayer ranking seems like a bad idea. It is not more difficult to beat two players than it is to beat one player, since it is a free for all game (and teaming up is practically not feasible). There is a factor that changes the game, though: bad neighbours. If a bot can easily conquer its neighbours, it will snowball and win the game. This means that we should play all possible player configurations (so, n! matches) to make this type of game fair.