Same skill, same rate, different results?

[marcolino]

In your example, bob and chris have the same starting skill (skill.mu = 25.0, skill.sigma = 25.0 / 3). They get the same rank, too (say, 2). If I call

trueskill.AdjustPlayers([bob, chris]);

I get two (sligtly) different results for their mu. Is this by design? Does the players order matter, when calculating the skills?

And, more: running your same example ("example.js"), with four players, bob and chris do get very different results for their mu and for their sigma, too... Is this by design? I would expect each player mu and sigma values to be only dependent on his/her previous skill and on his/her rank (players are not in "teams", right?)...

[freethenation]

How slightly? Floating point is used internally which will have slight rounding errors (http://floating-point-gui.de/basic/). Give me the full code for the first scenario.

I think example.js just has rounding issues as well. For my use case I just used three significant figures which always matched the calculator linked below. Also, trueskill is better then elo but far from perfect. It makes lots assumptions that are at best rough approximations (most obvious is that a players skill can be represented by a normal distribution). I suspect that assumptions made by trueskill will introduce more error then the numerical ones introduced by node-trueskill (I gave up on using trueskill because its assumptions did not fit my use case).

Microsoft has their code running at Rank Calculator. It is not open source but is a good sanity check so you can check if it is an issue with node-trueskill or an issue with your understanding of trueskill. I suspect your understanding is correct and its just that node-trueskill's implementation is not the best.

Rank Calculator
View on boson.research.mic... Preview by Yahoo

---

From: marcolino notifications@github.com To: freethenation/node-trueskill node-trueskill@noreply.github.com Sent: Monday, June 16, 2014 1:06 AM Subject: [node-trueskill] Same skill, same rate, different results? (#3)

In your example, bob and chris have the same starting skill (skill.mu = 25.0, skill.sigma = 25.0 / 3). They get the same rank, too (say, 2). If I call trueskill.AdjustPlayers([bob, chris]); I get two (sligtly) different results for their mu. Is this by design? Does the players order matter, when calculating the skills? And, more: running your same example ("example.js"), with four players, bob and chris do get very different results for their mu and for their sigma, too... Is this by design? I would expect each player mu and sigma values to be only dependent on his/her previous skill and on his/her rank (players are not in "teams", right?)... — Reply to this email directly or view it on GitHub.

[marcolino]

Sorry for my late answer... :-) Some differences are obviously from floating point round errors, you are right...

With this code:

var    bob = {};    bob.skill = [25.0, 25.0/3.0]
var  chris = {};  chris.skill = [25.0, 25.0/3.0]
bob.rank = 1;
chris.rank = 1;
trueskill.AdjustPlayers([bob, chris]);
console.log("bob:", bob.skill);
console.log("chris:", chris.skill);

I get:

bob:  [25.000000000000007, 6.513184555493111]
chis: [25.000000000000004, 6.513184555493111]

So, small dfferences for sigma values. My only doubt is: why the same operation ( I suppose the operation is the same for every player), with the same input, yelds different results? I knew about FP rounding errors, but also about their predictability... :-) However, this is not a real point... The real problem - as far as I can say - is with your "example.js":

var  alice = {};  alice.skill = [25.0, 25.0/3.0]
var    bob = {};    bob.skill = [25.0, 25.0/3.0]
var  chris = {};  chris.skill = [25.0, 25.0/3.0]
var darren = {}; darren.skill = [25.0, 25.0/3.0]
alice.rank = 1;
bob.rank = 1;
chris.rank = 2;
darren.rank = 2;
trueskill.AdjustPlayers([alice, bob, chris, darren]);
console.log("alice:", alice.skill);
console.log("bob:", bob.skill);
console.log("chris:", chris.skill);
console.log("darren:", darren.skill);

gives:

alice:  [28.19596431418999,  5.758099663992987]
bob:    [28.202592368691597, 5.754758392846645]
chris:  [21.79740763130838,  5.754758392846654]
darren: [21.804035685810483, 5.758099663993233]

So, quite big differences for sigma, both among winning players, alice and bob, and among loosing ones, chris and darren... Are these due to FP, too? Mu values instead are the same (excluding FP rounding errors) for first and last player, and for second and third player... very strange, to me... I would expect mu values to depend only on the number of matches played... Is this false?

You are right, for sure trueskill assumptions introduce by far bigger errors in the calculation, than FP... But, I need at least some "predictability"... :-) And, I think a normal distribution could be acceptable for me...

Thanks again for your attention...