search

sublee / trueskill

An implementation of the TrueSkill rating system for Python
https://trueskill.org/
Other
742 stars 112 forks source link

Implement a rank prediction function #24

Open bernd-wechner opened 6 years ago

bernd-wechner commented 6 years ago

I'm interested in how well TrueSkill performs at predicting match outcomes.

That's best done by comparing an actual result with a prediction TrueSkill would make based on current skills.

I can see from the factor graph that player skill is first translated to performance by injecting an uncertainty of Beta, and that these player performances are summed to calculate a team performance. I'm shaky on my reading of the graph at this point and exactly what math is behind adding Beta and adding performances, but what I can't find mentioned in the paper I'm reading is what the most likely match outcome is given a set of players in teams with known skills.

I have a feeling this is a fairly simple function. If it were a game of n individual players I imagine the predicted ranking is just the players ranked by their skill means (Mu values). Well, that would be my supposition anyhow.

It gets trickier in the general case of teams, and how to calculate a pranking prediction of a game that has n players distributed among m teams. I imagine needing to estimate team skill on the basis of team members, and might infer from the way performances are added to arrive at team performance, that skills can be added to arrive at team skill. But I'm not sure and still reading and getting my head around what it means to add two performances or skills (which Gaussian variables).

It's clearly not just adding the Gaussians, nor can I expect the mean (expected value) to be the sum of means (as then the net performance or skill of a team would grow with the number of members. It may of course prove to be the mean of the means or the weighted mean of means (taking into account partial play weightings) and that would not surprising given how elegantly Gaussians pan out in so many ways. Still I am hypothesizing and I see value in the package including a function that performs the prediction in the general team structure case (and perhaps to see it documented at trueskill.org).

Of course if anyone can provide any pointers that help me understand this I'm happy to try and nail it and implement it and PR it. But I'm floundering a little at the moment and so thought to drop a note here while I do.

rrjohnson commented 6 years ago

You mention you are reading a paper. What is the paper you are reading?

bernd-wechner commented 6 years ago

2007 TrueSkill(TM): A Bayesian Skill Rating System - Herbrich, Minka and Graepel