eVen-gits / EDH_matchmaker

8 stars 2 forks source link

Pod matching doesn't prioritize top seeds #2

Open fbatista opened 2 years ago

fbatista commented 2 years ago

To preserve tournament integrity, the matchmaking should prioritize the best matches to the top ranking players in the tournament, in the latest round.

If this doesn't happen, there can be an issue of pair-up / pair-downs happening in a way that more often impacts the tournament outcome. Because the players are shuffled and a pod is created by analysing the first player picked at random and getting the best pod for that player can lead to a situation where the worst ranked player is picked first and matched, when in reality that player should get a bye.


Also unrelated to this issue, is the scoring system. 3 points per win / bye leads to a situation where it's better for players to just draw the most possible games as possible instead of playing to win:

Given a 25% win rate, over 4 rounds, a player that wins 1 game and loses 3, gets 3 points A player drawing 4 games, gets 4 points.

eVen-gits commented 2 years ago

Thank you for your feedback! You are the first person to open an issue like this and it helps with keeping the discussion focused and accessible to everyone.

On the matter of matching order

    #POD making @ core.py, line 914
        random.shuffle(remaining)
        for p in sorted(remaining, key=self.tour.MATCHING, reverse=True):

Thank you for pointing this out.

As it stands, the remaining players are first shuffled and then sorted by matchmaking priority. Although it looks redundant, the shuffling part is there to randomize the matching order among those that have same score in terms of matching.

The matching metric has different scoring than the ranking.

    def MATCHING(_, x): return (
#highest priority for matching is received by players that received a bye (less games played)
        -x.games_played,
#after that, it will match players that have less players played. Ideally, all players  should have seen same number of opponents
        -x.unique_opponents,
#after that, it will pick the player according to their points, tiebreakers disregarded
        x.points,
#and if two players have same number of points, this is the "tiebreaker"
        -x.opponent_winrate
    )

I should really check if this is working as intended. But furthermore, we are working on creating an interface to test for "undesirable situations". I am doing this in collaboration with @Kahno and hopefully, we should have some results soon.

On the matter of final round matching

Currently, there is no defined number of rounds and new rounds can be created on demand. Usually, TOs have some special idea how to do this and for this reason, a manual way to create pods partially is offered. To do so, select players that you want to seat in a pod (ctrl+click) -> right click -> create pod. The remaining pods can still be auto generated by issuing the "Create pods" button. image

I agree that perhaps during tournament setup, a number of desired rounds could be provided and we could optimize the matching based on that. This seems like a good idea.

On the matter of scoring

This has been pointed out several times. As it stands, the software allows for custom scoring setup. A UI has been provided for this configuration specifically. image It is possible to configure the desired amount of points for win/draw/loss and additionally, the sizes of permitted pods and whether the tournament allows byes.

From my own perspective: I would rather see all players finish their games. A draw should generally also mean certain players have already been eliminated, although the round time does go up significantly during online play. I think the ratio between drawing and winning is OK from an individual player perspective - winning gives three times as many points as drawing. If all players were to agree to draw at the beginning, this is not really in the spirit of the competition and this would also only work for as long as everyone does this. As soon as one player "goes for the win", they will gain a big advantage with a single win.

Here's a counter-argument as well: Consider a tournament with 4 rounds. Two players go with the following win/draw/loss records: 1/0/1 and 0/3/1, meaning they have the same points. If the tournament is somewhat balanced and with the nature of the game the way it is, it's not unlikely for a player to get lucky and snag a win. Even the sitting order suggests this as going first has a higher probability for winning. Couple this with a good opening hand, and a player that's not considered a favourite gets three points for winning.

On the other hand, the other player, known to be consistend and very skilled is playing a stax deck. Because the rounds are short, the player draws often, although he could have won more matches if he had more time. The record 0/3/1 is formidable, as he was a contender for winning every round.

They have same points, but the "lucky" player still wins on tiebreakers. I think draws should still be respected, if not abused. I think it's more important to have good rules regarding slow play, rather than disregarding the draws.

Either way, this is highly specific to tournaments and tournament organizers' preferences. The option is there and everyone can tailor the scoring to their liking. I don't think there's a solution that works for everyone :)


Thank you again for raising your concerns. I hope I've addressed the issues you raise. Hopefully, we'll have the statistical evidence ready soon, as this is the highest priority for the software in my opinion. We still want the "confirmation" that everything is working the way we want, as it's impossible to assess empirically.

fbatista commented 2 years ago

Nice, this is actually a very nice write up. I'm not a tournament organizer, i'm a judge and i've worked on the MSTR / MSIPG docs. I'm also a software engineer, but unfortunately i don't have bandwidth to devote to helping out on a project like this, even more so given the fact that python is not my go-to language...

Anyway in terms of pairing algorithm, one of the things that has been discussed recently is the idea of using standings to define the starting player / pod seatings during swiss rounds, and make it so that top seeds are seated in 4th and bottom seed in 1st seat.

During single elimination rounds (after cut) the logic is reversed, as a way to provide a benefit for top seeded players during swiss.

Another thing that i haven't checked in this program regarding the algorithm, and that i usually mention: