Kender2 / swc-squadranking

SWC squad war ranking
http://squadsofdeath.com/squadranking
6 stars 8 forks source link

Replace TrueSkill #16

Open Kender2 opened 7 years ago

Kender2 commented 7 years ago

TrueSkill does not seem to work very well when the actual matchmaking is not done using it's results. It is also too hard for people to understand apparently.

The challenge is to come up with an alternative that does not have these problems:

Please discuss.

MidgetAl commented 7 years ago

I am not sure what I would recommend for a formula really. I do think it is tough. I do think some squads "game the system" which makes it easier to rise in the TrueSkill ranking, but I don't know the real fix.

The biggest issue for SOD to me is that it does not backtrack. For example, if I faced ByDesign in their first war, I didn't get the huge boost I would get if I faced them in MY first war and beat them. Also, we played and beat a team a long time ago that is now ranked in the top 50, we should get more credit, IMO, for being one of their few losses as they continue to win.

The next issue I think that matters is that it is not "up-to-date." I think of tennis when I say this. Just because Andy Murray had two or three years of struggling, does not mean he can't get all the way to the top later. I think if my squad rips off 20 wins in a row and 7 of them are against Brigade, ByDesign, Shadow Company, Imperial Snipers, etc. then I should be somewhere in the top 5 if not flat out number one, no matter what I did six months ago.

I know that all of this is theoretically taken into account in some ways. And I don't know what formula I would use exactly to change it. But I love that this clearly is your baby and you work hard for us.

Kender2 commented 7 years ago

From DeathStriker:

Kender - I don't (yet) want to create an account for github.com so I will comment here.

FIRST SOME CLARIFICATION OF TERMS(found here)

“Ranking” and “Rating” are often used interchangeably, but they are different concepts.

Rating = numerical approximation of player skill / probability of outcome predictor Ranking = numerical ordering of players among a pecking order (leaderboard, ladder, etc.) In short, rating is absolute, while ranking is relative. If the number one player in the world dies, everyone else gains +1 rank but does not change their rating.

ON TOPIC:

TrueSkill™, a skill-based ranking system developed by Microsoft for use with video game matchmaking on Xbox Live seems to be the "Gold Standard" for a system to be used to determine Rank in Multi-Player Games.

Currently there are several well known Player Ranking Systems:

ELO Ranking Glicko and Glicko-2 (a variant of ELO) TrueSkill (based upon Glicko) eGenesis Ranking (used in Tales in the Desert and also used in ChronX) It is interesting to note that for PvP Matchmaking the Game Developers were investigating using ELO (or some variant of such) as indicated by this Playdom Archived Thread (Scroll down and view Matchmaking System Improvements).

RECOMMENDATIONS:

Perhaps you should think about using ELO (or some variant). See GitHub Discussion about Multiplayer ELO.

Another recommendation would be an evaluation of using a Rating Percentage Index (RPI) which is how NCAA basketball, baseball, softball, hockey, soccer, lacrosse, and volleyball teams are ranked.

ADDITIONAL HELP:

The information found here can help in evaluating Ranking Systems other than TrueSkill. Reading the Web Article titled Rising from the Ranks: Rating for Multiplayer Games might also give you some ideas. Read about Ranking to get the "big picture".

Kender2 commented 7 years ago

Thanks DS. I am familiar with all these systems, and TrueSkill one is the one with the least drawbacks. The main problem is that these systems break down when the results are not used to do the actual matchmaking.

Maybe this simply can't be solved this way. An alternative would be to keep TrueSkill but divide the ranking up into multiple "tiers". We can fit 3-5 top 10s on the front page with link-throughs to the individual rankings. And then the next difficulty rears it's head: How to determine in which tier a squad belongs? Average basescore of the squad members might be a way. It's hard to come up with something that cannot be gamed by squads like 502sithregiment

I have some more ideas, but it's getting late..

MidgetAl commented 7 years ago

Gaming the system is always a major issue in these sort of cases. I am not exactly sure how to remedy it myslf.