SavanVaghela / ai-contest

Automatically exported from code.google.com/p/ai-contest
0 stars 0 forks source link

Add Elo confidence bounds to rankings and player profiles #188

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
Purpose of code changes on this branch:
Show elo confidence intervals to players.

When reviewing my code changes, please focus on:
Code correctness and display of information to user.

Screenshots of the changes to the rankings and profile can be seen at 
http://www.dropbox.com/gallery/9140666/1/screenshots?h=011b14

This change requires storing the confidence intervals reported by bayeselo in 
two new columns in the rankings table. I'm not sure if I should check in a new 
schema.sql to show this or if there is a better way. The required alter 
statements are:

ALTER TABLE rankings ADD plus_bound double default '1000' AFTER score;
ALTER TABLE rankings ADD minus_bound double default '1000' AFTER plus_bound;

Original issue reported on code.google.com by janzert on 5 Oct 2010 at 6:24

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
I think we only need to show a single +/- column because we are using Bayeselo 
in a way that the upper and lower confidence bound should be the same distance 
from the mean.

Alternatively, we might want to consider showing the actual upper and lower 
confidence bounds rather than their distances from the mean. That is, instead 
of <mean> <-> <+>, e.g. "2090 123 123" from the first row of the screenshot, we 
could show something like <-> <mean> <+>, e.g. "1967 2090 2213".

Original comment by Jake.McA...@gmail.com on 5 Oct 2010 at 3:14

GoogleCodeExporter commented 8 years ago
I've made the display changes suggest above. I agree that it is more 
informative this way. I also ended up changing the value stored in the database 
to the actual bound rather than the +/-. This seems generally more useful not 
only because that is the value we are always displaying now but also it will 
allow us to easily sort based on the lower bound later if desired.

Jake.McArthur has also worked on some javascript whisker plots that should be 
quite nice to add after this change goes in.

The new database alter statements are:

ALTER TABLE rankings ADD upper_bound double default '4000' AFTER score;
ALTER TABLE rankings ADD lower_bound double default '0' AFTER score;

Original comment by janzert on 5 Oct 2010 at 7:46

GoogleCodeExporter commented 8 years ago
I forgot to mention in the last comment. The reason bayeselo is currently 
giving symmetric bounds is because the accurate confidence interval calculation 
had been commented out since we weren't using them anyway before.

The command to calculate better intervals but with a still reasonably fast 
calculation is added back into this branch. The best interval calculation 
method is cubic with the number of participants (not games played though) so 
may be too expensive on the server. It would be nice if this could be tested on 
the server to see how long it takes in practice (see note in readgames.py).

Original comment by janzert on 5 Oct 2010 at 7:51

GoogleCodeExporter commented 8 years ago

Original comment by janzert on 6 Oct 2010 at 4:58

GoogleCodeExporter commented 8 years ago
Set review issue status to: WontFix

Original comment by janzert on 18 Nov 2010 at 7:03