search

pangafu / Hybrid_LeelaZero

Auto hybrid and test Leela Zero weights
https://github.com/gcp/leela-zero/issues/814
GNU General Public License v3.0
16 stars 5 forks source link

Test "hybrid" 6B weights on CGOS #4

Open pangafu opened 6 years ago

pangafu commented 6 years ago

https://github.com/gcp/leela-zero/issues/954

To see the true power of the hybrid, I hybrid 2 weights and run it on cgos.

  1. I make out 2 hybrid weight below: 6w24_081&aed&a91&92d&9ac&8ee&f66&4d5&5e4.zip 6w23_f66&081&9ac&8ee&4d5&5e4_1&0.5&0.5&0.5&0.5&0.5 (2).zip

W23 is inherited from No.86 weight (9ac646d7) and other 5 weight after it , W24 is inherited from No.86(9ac646d7) and No.87(92dd0397) and other 7 weights between them. And you can see the PO 200 match log in the zip file.

  1. Then I put run them on CGOS, the parameter is : LZ-HY-W23-PO1600: -g -p 1600 --noponder -t 1 -q -d -r 1 -w LZ-HY-W24-PO1600: -g -p 1600 --noponder -t 1 -q -d -r 1 -w LZ-HY-W24-NOLIMIT: -g --noponder -t 4 -q -d -r 1 -w

And there is many other leela zero weights run on cgos, such as LZ-0xx-p1600-t1-r1, StellarLZ-p6400, LeelaZero-test... I think LZ-HY-W23-PO1600 's parameter is same as LZ-0xx-p1600-t1-r1, so the result of them can compare.

  1. After 3 day's match, after 100 games, the elo of hybrid weights on cgos is close to static, so we can see the result (http://www.yss-aya.com/cgos/19x19/standings.html)

LZ-HY-W24-PO1600 3030
LZ-HY-W23-PO1600 3011

And other LZ-0XX-p1600-t1-r1 is not close to static(<100 games), the score of them is all bellow the 2 hybrid weights, and to my experiment, the score of LZ-0XX-p1600-t1-r1 will continue to drop during 50 to 100 games process.

So after several test, in my opinion:

  1. Hybrid can stronger the weights
  2. Hybrid seem can promote 50-80 cgos elo of original weights (according to 150-250 leela zero self elo)
  3. NOLIMIT is also get the best CGOS ELO of leela zero, so in the high playout game, "Hybrid" is also work.
  4. Hybrid can also get weaker weights, a stronger hybrid weight mix with an original weight, often get a weaker one.
killerducky commented 6 years ago

Hi, just a note that I'm the one running LZ-1a198f-t1-v2600. I guess you are running LZ-H6-W27-V2600? I posted a little about my plans here: https://www.reddit.com/r/cbaduk/comments/823ekq/lz1a198ft1v2600_on_cgos/

pangafu commented 6 years ago

@killerducky yes. I now start training a 20*128 network and hybrid for some test, now not run on CGOS.