lightvector / KataGo

GTP engine and self-play learning in Go
https://katagotraining.org/
Other
3.47k stars 563 forks source link

Katago 40b v100 progress on cgos #576

Open hwj-111 opened 2 years ago

hwj-111 commented 2 years ago

I plot the progress trend of 40b network based on the data on cgos beyes elo Roughly improved at +0.2 elo / network image

hwj-111 commented 2 years ago

Updated plot on 2022/02/10 image

hwj-111 commented 2 years ago

I found now cgos BayesElo has accumulates enough b40v50 data points. So I plot both v100 and v50 together. v50 fitted line (slope ~ 0.15) seems lower by 180 elo than the v100 fitted line (slope ~ 0.19). This value seems a bit larger than the benefit of purely doubling the visits (+125 elo), indicating v100 is much preferred than v50 when using 40b networks (I guess, more visits needed before reach the 40b saturated limit)

image

hwj-111 commented 2 years ago

Here is the plot of v100 (red) and v50 (green) of 60b network progress. The delta between them is about 170elo. The slope is much smaller than 40b ( only about ~0.14 per network# for v100, and 0.05 for v50!), indicating the progress is slower (when evaluate progress on v100). Maybe, v200 or v400 is much better ( large slope)?

image

hwj-111 commented 1 year ago

I have ran some katago 60b nets on CGOS with visits/move = 200 (blue dots). Their progress is clearly better than v=100 (red) and 50 (green). Maybe, to see the actual progress, visits/move has to be higher, at least, 200 or so

image

hwj-111 commented 1 year ago

update for katago 60b progress on CGOS, with vistts/move = 200, 100, and 50 image