glandfried / TrueSkillThroughTime

TrueSkill Through Time: the Julia, Python and R packages.
25 stars 3 forks source link

Can TSTT model continuous outcomes and weights for each sample? #1

Closed EvanZ closed 3 years ago

EvanZ commented 3 years ago

An obvious use case for something like this would be modeling basketball player skills. In each "stint" of a game there are 10 players, 5 on each side. A stint may last anywhere from a few to a dozen or so possessions where each team has a chance to score. At the end of the stint each side has scored some number of point and the difference between the two sides can be considered an outcome or result (for example +4 indicates the home team scored 4 more points than the visiting team in a stint). So I'm wondering if there is a way with the current model to a) handle this kind of outcome that is not just a 1 or a 0 and b) handle the weighting implied by varying number of possessions in each stint (eg longer stints should be more meaningful). I guess another way of saying (b) is that it is like a "partial" game or sub-game, if that makes sense. Thanks!

glandfried commented 3 years ago

Evan Zamir,

a) Continuous outcomes.

We plan to implement the model [1] soon, but it is not yet available.

[1] Score-based Bayesian Skill Learning: https://www.microsoft.com/en-us/research/wp-content/uploads/2012/01/sbsl_ecml2012.pdf

b) Weights

Weights are already implemented in Julia, and will soon be incorporated into the Python and R package.

Be careful with mathematics when using weights, because we do not obtain the same result if we instantiate one player participating in the whole game (100%), than one player participating in the whole game but in two parts (e.g. 50% + 50%). It is a problem of the model, not of the implementation. Using sublee's impementation:

import trueskill as ts
import math as m
env = ts.TrueSkill(tau=0.0, draw_probability=0.0, beta=1, mu=2, sigma=6)
a = env.Rating()
b = env.Rating()
(a1), (_), = env.rate([[a],[b]])
(a2), (_, _), = env.rate([[a],[b,b]], weights=[[1],[0.5,0.5]])
(a3), (_, _), = env.rate([[a],[b,b]], weights=[[1],[m.sqrt(0.5),m.sqrt(0.5)]])
print(a1, a2, a3)

trueskill.Rating(mu=5.339, sigma=4.985)        
trueskill.Rating(mu=5.856, sigma=4.597)
trueskill.Rating(mu=5.600, sigma=4.949)

Mathematically there is no weight that recovers a team performance with the same mean and standard deviation. Here why.

weights

EvanZ commented 3 years ago

Thank you for that thorough explanation!