Player stuck at ELO score of 0

[fimpfimp]

We have installed PingPong-bot in coinify-com.slack.com . It worked great for a while, however now one of the players, "stj", has got stuck with an ELO rating of 0. No matter if he wins or loses many games his rating remains 0.

Bug?

[fimpfimp]

Actually it seems all players are stuck with their ELO rating, not just one player.

[dblock]

Interesting. The data has correct ELOs, so it must be a display problem. Can you show me where - is this in the output from leaderboard or rank?

[fimpfimp]

This is when showing the leaderboard.

[fimpfimp]

User "lse" is not stuck I see. So SOME users are stuck with their ELO rating.

[dblock]

Can you copy-paste output here? Feel free to doctor it if needed.

[fimpfimp]

pong leaderboard 20

PlayPlay.io - Ping-Pong for SlackBOT    [17:41]  
1. jbo: 81 wins, 41 losses (elo: 105, lws: 8)
2. mbn: 89 wins, 72 losses (elo: 33, lws: 7)
3. jpe: 83 wins, 68 losses (elo: 24, lws: 6)
4. mbe: 1 win, 1 loss (elo: 2)
5. stj: 83 wins, 89 losses (elo: 0, lws: 6)
6. lhu: 62 wins, 68 losses (elo: -1, lws: 7)
7. vvm: 5 wins, 7 losses (elo: -22, lls: 3)
8. tny: 2 wins, 3 losses (elo: -25)
9. rfa: 2 wins, 3 losses (elo: -50, lls: 3)
10. jak: 1 win, 2 losses (elo: -53)
11. gko: 2 wins, 4 losses (elo: -65)
12. lbo: 6 wins, 11 losses (elo: -80, lls: 4)
13. lse: 33 wins, 81 losses (elo: -108, lls: 7)

[dblock]

So when you record a game elo's aren't moving? Or just stj's? Note that only users that play see their elo affected. Did those other users play and that wasn't recorded? Is it just elo or also the numbers? It would be helpful to get a recent game from your slack history with elos before/after and whatever messages/conversations around that.

[fimpfimp]

I just recorded a loss against stj. My ELO went down, his stayed at 0:

Lasse Birk Olesen   [17:41]  
pong leaderboard 20

PlayPlay.io - Ping-Pong for SlackBOT    [17:41]  
1. jbo: 81 wins, 41 losses (elo: 105, lws: 8)
2. mbn: 89 wins, 72 losses (elo: 33, lws: 7)
3. jpe: 83 wins, 68 losses (elo: 24, lws: 6)
4. mbe: 1 win, 1 loss (elo: 2)
5. stj: 83 wins, 89 losses (elo: 0, lws: 6)
6. lhu: 62 wins, 68 losses (elo: -1, lws: 7)
7. vvm: 5 wins, 7 losses (elo: -22, lls: 3)
8. tny: 2 wins, 3 losses (elo: -25)
9. rfa: 2 wins, 3 losses (elo: -50, lls: 3)
10. jak: 1 win, 2 losses (elo: -53)
11. gko: 2 wins, 4 losses (elo: -65)
12. lbo: 6 wins, 11 losses (elo: -80, lls: 4)
13. lse: 33 wins, 81 losses (elo: -108, lls: 7)

Lasse Birk Olesen   [18:29]  
pong lost to stj

PlayPlay.io - Ping-Pong for SlackBOT    [18:29]  
Match has been recorded! stj defeated lbo.
http://media3.giphy.com/media/ORM4TGTrZe74I/giphy.gif (1MB)

Lasse Birk Olesen   [18:29]  
pong leaderboard 20

PlayPlay.io - Ping-Pong for SlackBOT    [18:29]  
1. jbo: 81 wins, 41 losses (elo: 105, lws: 8)
2. mbn: 89 wins, 72 losses (elo: 33, lws: 7)
3. jpe: 83 wins, 68 losses (elo: 24, lws: 6)
4. mbe: 1 win, 1 loss (elo: 2)
5. stj: 84 wins, 89 losses (elo: 0, lws: 6)
6. lhu: 62 wins, 68 losses (elo: -1, lws: 7)
7. vvm: 5 wins, 7 losses (elo: -22, lls: 3)
8. tny: 2 wins, 3 losses (elo: -25)
9. rfa: 2 wins, 3 losses (elo: -50, lls: 3)
10. jak: 1 win, 2 losses (elo: -53)
11. gko: 2 wins, 4 losses (elo: -65)
12. lbo: 6 wins, 12 losses (elo: -102, lls: 4)
13. lse: 33 wins, 81 losses (elo: -108, lls: 7)

[dblock]

Found the problem. In your case stj has played so many games that their elo is not affected by either a win or a loss against lbo, who played very few games. The math says that their elo should be moved by 0.004 while the opponent's elo moves by 35.7. We round up.

I am not sure whether I should be fixing this by rounding elo changes to 1, that's not really "true". Thoughts?

[fimpfimp]

I think Elo score changes are normally not affected by the amount of games played before?

I.e. the "Elo Rating Change Calculator" here does not take amount of previous games as input: http://www.3dkingdoms.com/chess/elo.htm

If you are weighing amount of games previously played I don't think this can actually be called an Elo score?

[fimpfimp]

Same with the calculator provided by FIDE: https://ratings.fide.com/calculator_rtd.phtml

[dblock]

This is called elo decay and was taken slack-pongbot, this bot was a port. I believe it that from League of Legends, maybe @andrewvy can comment? (he introduced it in https://github.com/andrewvy/slack-pongbot/blob/master/lib/pong.js#L248).

I am not saying we should keep it, but I like that elo moves differently for a player who plays a lot vs. a player who plays little.

I guess I am saying I don't know what the right answer is just yet. It would be helpful if someone (@fimpfimp?) made a PR that replaces the current elo math by standard out of the box elo math. Ideally we want to provide different elo strategies anyway, I could code something that allows you to select that.

[dblock]

Failing test for this issue in https://github.com/dblock/slack-gamebot/tree/issue-136.

[fimpfimp]

Here's another example from stj, playing a game gainst two opponents who also have many games:

stj [16:43]  
pong leaderboard infinity

PlayPlay.io - Ping-Pong for SlackBOT    [16:43]  
1. jbo: 82 wins, 41 losses (elo: 105, lws: 9)
2. mbn: 90 wins, 73 losses (elo: 32, lws: 7)
3. jpe: 83 wins, 69 losses (elo: 23, lws: 6)
4. mbe: 1 win, 1 loss (elo: 2)
5. stj: 84 wins, 90 losses (elo: 0, lws: 6)
6. lhu: 62 wins, 69 losses (elo: -1, lws: 7)
7. vvm: 5 wins, 7 losses (elo: -22, lls: 3)
8. tny: 2 wins, 3 losses (elo: -25)
9. rfa: 2 wins, 3 losses (elo: -50, lls: 3)
10. jak: 1 win, 2 losses (elo: -53)
11. gko: 2 wins, 4 losses (elo: -65)
12. lbo: 6 wins, 12 losses (elo: -102, lls: 4)
13. lse: 35 wins, 81 losses (elo: -104, lls: 7)

lse [16:43]  
pong lost to @mbn @lhu with @stj

PlayPlay.io - Ping-Pong for SlackBOT    [16:43]  
Match has been recorded! mbn and lhu defeated lse and stj.
http://media4.giphy.com/media/XV2fO0nlD0XPG/giphy.gif (154KB)

stj [16:43]  
pong leaderboard infinity

PlayPlay.io - Ping-Pong for SlackBOT    [16:43]  
1. jbo: 82 wins, 41 losses (elo: 105, lws: 9)
2. mbn: 91 wins, 73 losses (elo: 32, lws: 7)
3. jpe: 83 wins, 69 losses (elo: 23, lws: 6)
4. mbe: 1 win, 1 loss (elo: 2)
5. stj: 84 wins, 91 losses (elo: 0, lws: 6)
6. lhu: 63 wins, 69 losses (elo: 0, lws: 7)
7. vvm: 5 wins, 7 losses (elo: -22, lls: 3)
8. tny: 2 wins, 3 losses (elo: -25)
9. rfa: 2 wins, 3 losses (elo: -50, lls: 3)
10. jak: 1 win, 2 losses (elo: -53)
11. gko: 2 wins, 4 losses (elo: -65)
12. lbo: 6 wins, 12 losses (elo: -102, lls: 4)
13. lse: 35 wins, 82 losses (elo: -104, lls: 7)

[dblock]

Yep. I haven't decided what to do with this. Probably simple elo is the way to go, but I'd like to implement strategies.

You can always start a new season in the meantime.

[fimpfimp]

Ok. Though seasons is not available in the free version. Would you be willing to give us an upgrade so we can continue to use the leaderboard? :)

[dblock]

As a rule of thumb I give free upgrades only to those contributing code, sorry. I feel like Coinify could afford a subscription and support this open-source project too.

[jordanhudson]

1. pavel: 31 wins, 13 losses, 4 ties (elo: 1393, lws: 8)
2. lucas: 21 wins, 13 losses, 1 tie (elo: 1337, lws: 6)
3. ted: 38 wins, 25 losses, 5 ties (elo: 1315, lws: 10)
4. dustin: 40 wins, 23 losses, 3 ties (elo: 1307, lws: 5)
5. evan.elder: 7 wins, 4 losses, 1 tie (elo: 1302, lws: 3)
6. jm: 77 wins, 45 losses, 3 ties (elo: 1233, lws: 7)
7. jhud: 74 wins, 79 losses, 6 ties (elo: 1200, lls: 7)
8. roberto: 48 wins, 55 losses, 4 ties (elo: 1191, lws: 5)
9. paul: 4 wins, 5 losses (elo: 1168, lws: 3)
10. phil: 5 wins, 7 losses (elo: 1159, lls: 6)
11. jonathan: 14 wins, 19 losses (elo: 1151, lws: 6)
12. jamie.heale: 7 wins, 19 losses, 1 tie (elo: 1106, lls: 11)
13. craig: 16 wins, 41 losses, 2 ties (elo: 1077, lls: 7)
14. neil: 7 wins, 19 losses (elo: 1028, lls: 11)
15. kb: 3 wins, 28 losses, 1 tie (elo: 933, lls: 18)

jhud:  chess lost to roberto

1. pavel: 31 wins, 13 losses, 4 ties (elo: 1393, lws: 8)
2. lucas: 21 wins, 13 losses, 1 tie (elo: 1337, lws: 6)
3. ted: 38 wins, 25 losses, 5 ties (elo: 1315, lws: 10)
4. dustin: 40 wins, 23 losses, 3 ties (elo: 1307, lws: 5)
5. evan.elder: 7 wins, 4 losses, 1 tie (elo: 1302, lws: 3)
6. jm: 77 wins, 45 losses, 3 ties (elo: 1233, lws: 7)
7. jhud: 74 wins, 80 losses, 6 ties (elo: 1200, lls: 7)
8. roberto: 49 wins, 55 losses, 4 ties (elo: 1193, lws: 5)
9. paul: 4 wins, 5 losses (elo: 1168, lws: 3)
10. phil: 5 wins, 7 losses (elo: 1159, lls: 6)
11. jonathan: 14 wins, 19 losses (elo: 1151, lws: 6)
12. jamie.heale: 7 wins, 19 losses, 1 tie (elo: 1106, lls: 11)
13. craig: 16 wins, 41 losses, 2 ties (elo: 1077, lls: 7)
14. neil: 7 wins, 19 losses (elo: 1028, lls: 11)
15. kb: 3 wins, 28 losses, 1 tie (elo: 933, lls: 18)

[jordanhudson]

Couple options here:

1) Seems there's some rigidity in place, where the more you've played, the more rigid your elo becomes, and it becomes so rigid that it doesn't move. Put some sort of cap on that rigid part of the equation.

2) Start storing people's ELO rating with a decimal, and do the increment/decrement as a decimal.

3) Just hardcode the minimum elo change as 1.

[dblock]

I think (2) would be a safe bet, and generally we could start showing elo with a decimal. I'd take PRs!

[jordanhudson]

the problem with that though.... maybe the rigidity is infinite? so as people approach 1000 games played, are they going to be altering their elo by 0.0000001 and stuff? That seems bad.

[jordanhudson]

I feel like (1) is the best solution, and maybe (3) is the value solution (very small amount of code for how much positive effect is gained)

[jordanhudson]

unless of course there IS an end to the rigidity slope, and 0.000001 increments are not a worry

[andrewvy]

^ Ideally, it would be nice to add time decay. Rigidity is in place so you can't gain a massive amount of elo in a short amount of time. Right now, that rigidity increases permanently for every game (and clears based on season).

It would be nicer to have the rigidity decay when the player hasn't recently been playing games. (As your skill level naturally decays a bit after not playing for a while too..) So, after a set amount of time of non-play, the player should be back to normal elo gain. 😄

[jordanhudson]

i like the rigidity. i just have an inkling it has no cap. like, elo gain of 0.1 or 0.01 says to me the rigidity is out of control. we play chess every day. sometimes multiple times per day.

[Ted-Bench]

Chess.com uses Glicko system instead of the Elo system for a lot of these reasons:

https://www.chess.com/article/view/chess-ratings---how-they-work

[dblock]

Maybe a fix? https://github.com/dblock/slack-gamebot/compare/master...frederikkuehn:master