FAForever / server

The servercode for the Forged Alliance Forever lobby
http://www.faforever.com
GNU General Public License v3.0
66 stars 61 forks source link

Lobby optimal balance issues (gray rating matchmaking) #684

Closed Draiget closed 4 years ago

Draiget commented 4 years ago

Hi folks, Dunno it this is right repo to make issue in, otherwise please correct me which one I should move it, thanks!

We had a game with friends today with "Optimal balance" set in lobby, but looks like it distributed players absolutely unbalanced - 238 vs 2412. That probably because we're all gray, but still, why don't balance gray players depending on their unset rating like white ones? It looks like a way to farm it.

P.S. Replay id is - #12845030. P.P.S. Any way to drop/rollback that game rating change for both teams?

Crotalus commented 4 years ago

What do you expect when there are grey players in the game? A 0 rated player with almost no games will be impossible to predict rating for. There probably was no combination of players that would give a high quality score (trueskill wise).

In this case you probably should have balanced manually, or maybe just go complete random.

And why rollback? Keep playing games and you'll see that your ratings oscillate to a more correct prediction with time. Rolling back will just slow down that progress...

Draiget commented 4 years ago

What do you expect when there are grey players in the game?

Game still show rating for gray player, not final, but why don't balance by that number? In this particular game zero players can be balanced with 1000+, and it will be more optimal than it actually did, why not?

Crotalus commented 4 years ago

Trueskill has two values, mu (perceived skill) and sigma (variance). Grey colors means high sigma, high uncertainty. The visible rating is then something like mu - 3*sigma, that is the "most pessimistic" rating of a player.

So in this case with 0 rated players, they start at 1500 perceived rating and then 500 sigma which means 0 in visible rating. This could be valued about same as someone with 500 visible rating (maybe has something like 1000 perceived rating - 3*170 sigma)

When I watched replay I see that the 0 rated players were complete noobs so I can agree that it failed quite hard. But imagine if one of these 0 rated players actually had played the game (or a similar) before, then it could have looked different.

The rating of grey players move up / down very fast in the beginning so it finds the "true rating" more quickly, so after a couple of games it should get better and better at predicting teams.

https://en.wikipedia.org/wiki/TrueSkill

Draiget commented 4 years ago

But imagine if one of these 0 rated players actually had played the game (or a similar) before, then it could have looked different.

Sounds reasonable, but I think there's not too much players who will be able to learn how to play within one game and deep dive to it. It wouldn't be better to mark 0-games people as real zero for the balancing system? That might help in cases like this.

Anyway, thank you for the explanation. Just very much surprised such behavior of the "Optimal balance".

Askaholic commented 4 years ago

You can't expect an algorithm to be able to optimally balance a game, when there is not enough data on player's skill. Your best bet in that situation is to balance manually using your external knowledge of the players experience.

Gonna close this as it's: