vivekjoshy / openskill.py

Multiplayer Rating System. No Friction.
https://openskill.me
MIT License
264 stars 13 forks source link

let score difference be reflected in rating #55

Closed jonathan-scholz closed 1 year ago

jonathan-scholz commented 2 years ago

When you enter scores into rate(), the difference between the scores have no effect on the rating - meaning: rate([team1,team2],score(1,0)) == rate([team1,team2],score(100,0)) is true. They have exactly the same rating effect on team1 and team2.

I don't know if it is mathematical possible and how it would look like. But it would be great if the difference could be somehow factored into the calculation, as it is (if your game has a score) quite an important datapoint for skill evaluation.

vivekjoshy commented 2 years ago

I don't know if it is mathematical possible and how it would look like. But it would be great if the difference could be somehow factored into the calculation, as it is (if your game has a score) quite an important datapoint for skill evaluation.

I'm not aware of any papers that successfully implement score margins for any of the models in this library. There is a 2020 paper by Kovalchik that does it somewhat successfully for Elo with 4 different MOV models, but I'm not aware of how to adapt it to the Weng-Lin models.

Sources:

  1. Kovalchik, S. (2020). Extension of the Elo rating system to margin of victory. International Journal of Forecasting, 36(4), 1329–1341. https://doi.org/10.1016/j.ijforecast.2020.01.006
philihp commented 1 year ago

I think this is out of scope, but don't let me stop you. You'll want to create a new model and benchmark it against the other models which are naive to score margins.

jonathan-scholz commented 1 year ago

Thank you @daegontaven for the hint to the resource. I was looking for a paper like this but couldn't find it. Let's see, maybe I can come up with something.

jonathan-scholz commented 1 year ago

After looking into some papers it seems, this is not easily implemented. It depends a lot on the specific game/sport and needs to be fitted before applicable. I will write something for my use case, but not a general solution for the package.