large change in rating variability / certainty

[echo-bravo-yahoo]

this change: https://github.com/nemasu/puddle-farm/commit/795632d28347c8a29fadcf434f2278faa11371a5#diff-8b8f70eea0b77f60fbcaf8a1def43908de41fe747cdfd74645742b7533a859bfL464

changed the variability algorithm from:

let mean_a_new = mean_a + direction_of_update * 10.0 * (result_suprise + 0.001) * sigma_a / f64::sqrt(sqrt_match_variablity);

to:

let mean_a_new = mean_a + direction_of_update * 10.0 * (result_suprise + 0.001) * f64::sqrt(sigma_a / sqrt_match_variablity);

was this intended? is there any documentation on the algorithm used?

looking at wiki, it appears that this (Eb) may be the inspiration, but it's still a little divergent from both the old and new score formulas in puddle-farm.

[nemasu]

Hello, yes this was intended.

I'm not actually the dev for this rating algorithm, but the change was explained to me like this: "the purpose of the square root change is to make the behavior of the variance adjustment more consistent in games where the two players have very different variances".

is there any documentation on the algorithm used?

At the moment, just the comments in the function itself.

[echo-bravo-yahoo]

would you be interested in writing (or reviewing/accepting) a PR that adds some units tests to demonstrate how the variance/scoring algorithm treats some sample game histories? the new & old variance algorithms are very different (players going from +/-70 to +/-2, for instance), and it seems like it'd be more predictable to either standardize on the chess standard/algorithm or hide variance?

[nemasu]

I don't mind adding unit tests, but for any algorithm changes, it would be best to discuss it with the designer.

[echo-bravo-yahoo]

gotcha! who should we involve in this discussion about the algorithm?

[nemasu]

I've let them know, will reply soon I think.