freethenation / node-trueskill

JavaScript implementation of TrueSkill player ranking
Apache License 2.0
48 stars 5 forks source link

Are you supposed to be able to use the SetParameters function? #4

Closed kaamodt closed 9 years ago

kaamodt commented 9 years ago

First of all, thanks for this implementation. Saved me a lot of time :)

I am using TrueSkill ranking system with a game without any possibility for a draw. For this reason I wanted to set the draw possibility to 0%. Even through your README does not say anything about the SetParameters function, I had read enough about the system before I found this implementation that I knew it should be possible to set the draw possibility. I used it like this trueskill.SetParameters((25/6.0, null,0.0, 25/300.0)); before used the AdjustPlayers function.

This gave me some strange behavior. A player with a high uncertainty gets a way too low uncertainty after 1 game.

BEFORE 
[ { player_id: 'mDCFDQkY2GGgMKzvQ', skill: [ 25, 8.333333333333334 ], rank: 2 },
 { player_id: 'eXNBaTc2f5KSjJykh',  skill: [ 25.72443324433869, 0.20934393495292025 ], rank: 5 },
{ player_id: 'aJPnuk6imnbvuADNM', skill: [ 26.16025584007285, 0.2355471417862665 ],  rank: 1 },
 { player_id: 'qxHymPTihydrgfGLL',  skill: [ 25.32898129643673, 0.22888298608451718 ],  rank: 4 },
{ player_id: 'eNeveqzNJ6BmJRwtA',  skill: [ 25.50360240460169, 0.2139260880183892 ], rank: 3 } ]
AFTER 
[ { player_id: 'mDCFDQkY2GGgMKzvQ',skill: [ 26.028519380002862, 0.3096701184461979],rank: 2 },
 { player_id: 'eXNBaTc2f5KSjJykh',skill: [ 25.296387392372623, 0.20745832019105673 ], rank: 5 },
{ player_id: 'aJPnuk6imnbvuADNM',skill: [ 26.38941060209121, 0.2827387205322086 ], rank: 1 },
{ player_id: 'qxHymPTihydrgfGLL', skill: [ 25.43866605180992, 0.19646251033747672 ], rank: 4 },
{ player_id: 'eNeveqzNJ6BmJRwtA', skill: [ 25.652680196361583, 0.20376005119680646 ], rank: 3 } ]

You can see that the first player has an uncertainty of 8.3 before the recalculation and only 0.3 after. This seems way to low. I took me awhile, but when I figured out it might be the SetParameters call I tried without and the after uncertainty ended up at 4.5. This seems more correct and is aligned to the result I get from the TrueSkill calculatar that microsoft has.

So, as the title stated, can I use the SetParameters function in my code?

freethenation commented 9 years ago

Its not in my example but you definitely can. If you read portedTrueskill.coffee there is a long comment about the function:

Sets three global parameters used in the TrueSkill algorithm. beta is a measure of how random the game is. You can think of it as the difference in skill (mean) needed for the better player to have an ~80% chance of winning. A high value means the game is more random (I need to be much better than you to consistently overcome the randomness of the game and beat you 80% of the time); a low value is less random (a slight edge in skill is enough to win consistently). The default value of beta is half of INITIAL_SIGMA (the value suggested by the Herbrich et al. paper). epsilon is a measure of how common draws are. Instead of specifying epsilon directly you can pass draw_probability instead (a number from 0 to 1, saying what fraction of games end in draws), and epsilon will be determined from that. The default epsilon corresponds to a draw probability of 0.1 (10%). (You should pass a value for either epsilon or draw_probability, not both.) gamma is a small amount by which a player's uncertainty (sigma) is increased prior to the start of each game. This allows us to account for skills that vary over time; the effect of old games on the estimate will slowly disappear unless reinforced by evidence from new games.

In general I would recommend you test anything you write against Microsoft's calculator to 1) make sure you understand whats going on and 2) make sure this port/implementation is numerically stable enough for your purposes (this implementation uses JavaScripts floating point so you may run into rounding issues).

kaamodt commented 9 years ago

Thanks for your answer.

I read the comment before I posted.

However, after reviewing my code again now I saw an error in my code. I had an extra set of parentheses around the parameters so I was passing in (25/6.0, null, 0, 25/300.0) as the Beta value instead of all the values. Oh man, do I feel like a retard now....

The package seems to work perfectly :) Thanks for making it.