search

sublee / trueskill

An implementation of the TrueSkill rating system for Python
https://trueskill.org/
Other
742 stars 112 forks source link

Documentation convergence speed #37

Open quancore opened 5 years ago

quancore commented 5 years ago

Hi, I am writing my master thesis and there is an issue about the convergence of TrueSkill in our case. We have thousands of players with a different number of matches and we want to only consider players that TrueSkill approximates real skill of a player 'well'. Because we are using mu values of players as data labels in a supervised learning task, so it is important that the labels should be in good quality. Regarding the issue, I have several questions:

  1. What is the best measure to evaluate the convergence of TrueSkill?
  2. If the answer is the number of matches, we found a table in the documentation indicates, for 4:4 game, 40 matches is needed to the convergence of TrueSKill. What is the source of information (or how the calculation of this magic number done)?
  3. We thought that the sigma value of a player can also indicate the quality of convergence. In which sigma threshold appr. sure convergence of skill is good?

Thanks in advance for all guys contribute the issue.

sublee commented 5 years ago

Hi,

I think your question is out of the project's boundary. I recommend you asking to Microsoft Research who invented TrueSkill instead.

quancore commented 5 years ago

At least can you give a source where you found or calculated the convergence table in TrueSkill doc.

quancore commented 5 years ago

Found answer: it is in the Microsoft lab page in TrueSKill. You can still keep open a while, maybe someone can give an idea about the problem.

sublee commented 5 years ago

@quancore Oh, I just remembered the convergence chart in the paper. I had to mention it for you but I supposed you've already read it.

If you didn't read Jeff Moser's article yet, it also will be helpful to you. When I When I implemented TrueSkill, it was very helpful and more friendly to me.