search

CuriosAI / sai

SAI: a fork of Leela Zero with variable komi.
GNU General Public License v3.0
105 stars 11 forks source link

sai79 v400 vs lz250 v1 #47

Open l1t1 opened 4 years ago

l1t1 commented 4 years ago
net games win
77 48 11
80 36 5
82 74 24
83 38 6
86 45 9
87 46 10
89 36 5
101 54 14
102 91 33
104 77 26
105 66 20
107 46 10
108 70 22
110 41 7
111 95 35
113 56 15
114 70 22
119 74 24
120 102 39
121 77 26
122 83 29
124 68 21
125 36 5
126 54 14
127 72 23
129 131 54
139 278 131
165 108 42
166 236 109
169 466 230

validation -k sai77_lz250 -n af4.gz -o "-g -v 400 -r 5 -w" -n networks/3d415846183a7f51a40aaa80a007d886668e759c37be8712febf09ec2823d257.gz -o "-g -v 1 -r 5 -w" -- sai -- sai

11 wins, 37 losses
The first net is worse than the second
af4.gz   v networks ( 48 games)
              wins        black       white
af4.gz     11 22.92%    5 21.74%    6 24.00%
networks   37 77.08%   18 78.26%   19 76.00%
                       23 47.92%   25 52.08%
l1t1 commented 4 years ago

http://www.yss-aya.com/cgos/19x19/standings.html

Game Program Name Rating Games Played  
— SAI_204_d8bf_p4000 2746 193
— SAI_238_40c4_p4000 2715 140
641525 LZ_258_32e8_r1_p1 2708 259
— LZ_204_05d1_r1_p1 2693 15930

SAI_238_40c4_p4000

LZ_258_32e8_r1_p1 2710 10 / 17 58.82
LZ_204_05d1_r1_p1 2699 1 / 2 50.00

SAI_204_d8bf_p4000

LZ_204_05d1_r1_p1 2690 9 / 14 64.29
nanzi commented 4 years ago

I'll make a larger scale of matches as you referred ogs games.

nanzi commented 4 years ago

./validation.exe -g2 -k ../PkAutoSave/lz258s_sai200/ -n 40b.gz -o "-g -v 1 -r 5 -w" -n networks/3f7781640d0e691cf3a3fd4d1d02462e87b81910395440621e81109b2cd59f76.gz -o "-g -v 400 -r 5 -w" -- sai -- sai

42 wins, 66 losses
The first net is worse than the second
40b.gz   v networks ( 108 games)
              wins        black       white
40b.gz     42 38.89%   23 39.66%   19 38.00%
networks   66 61.11%   35 60.34%   31 62.00%
                       58 53.70%   50 46.30%
l1t1 commented 4 years ago

./validation.exe -g2 -k ../PkAutoSave/lz258s_sai200/ -n 40b.gz -o "-g -v 1 -r 5 -w" -n networks/3f7781640d0e691cf3a3fd4d1d02462e87b81910395440621e81109b2cd59f76.gz -o "-g -v 400 -r 5 -w" -- sai -- sai

42 wins, 66 losses
The first net is worse than the second
40b.gz   v networks ( 108 games)
              wins        black       white
40b.gz     42 38.89%   23 39.66%   19 38.00%
networks   66 61.11%   35 60.34%   31 62.00%
                       58 53.70%   50 46.30%

could you post a list of all score of sai 200,210...

l1t1 commented 4 years ago

validation -g 1 -k sai250_lz250 -n d8b.gz -o "-g -v 400 -r 5 -w" -n lz258.gz -o "-g -v 1 -r 5 -w" -- sai -- sai

Stopping engine.
3 wins, 1 losses
4 games played.
Status: 0 LLR 0.232798 Lower Bound -2.94444 Upper Bound 2.94444
Engine has started.
time_settings 0 1 0
Starting GTP commands sent.
*ERROR*: Could not talk to engine after launching.
nanzi commented 4 years ago

./leela-zero-0.17-win64/validation.exe -g 2 -k PkAutoSave/v1/lz258s_sai250/ -n weights/40b.gz -o "-g -v 1 -r 5 -w" -n sai-0.17.5-gpu/networks/d8b59d6c546b3aae938f8c4de5668fc73d4b8b01a22a4c9e9cb026114897297f.gz -o "-g -v 400 -r 5 -w" -- sai -- sai


46 wins, 70 losses
The first net is worse than the second
weights/ v sai-0.17 ( 116 games)
              wins        black       white
weights/   46 39.66%   29 41.43%   17 36.96%
sai-0.17   70 60.34%   41 58.57%   29 63.04%
                       70 60.34%   46 39.66%

./leela-zero-0.17-win64/validation.exe -g 2 -k PkAutoSave/v1/lz258s_sai260/ -n weights/40b.gz -o "-g -v 1 -r 5 -w" -n sai-0.17.5-gpu/networks/5bb4703de9e2139a7f71ab65cea330ea090da6820d24b29e6c6646e860592a02.gz -o "-g -v 400 -r 5 -w" -- sai -- sai


62 wins, 84 losses
The first net is worse than the second
weights/ v sai-0.17 ( 146 games)
              wins        black       white
weights/   62 42.47%   34 43.04%   28 41.79%
sai-0.17   84 57.53%   45 56.96%   39 58.21%
                       79 54.11%   67 45.89%

./leela-zero-0.17-win64/validation.exe -g 2 -k PkAutoSave/v1/lz258s_sai267/ -n weights/40b.gz -o "-g -v 1 -r 5 -w" -n sai-0.17.5-gpu/networks/8ce00e8a2790c7278401a121bcf89b0e63cd3c5e6419e25b5bab36f3b49fe295.gz -o "-g -v 400 -r 5 -w" -- sai -- sai

25 wins, 50 losses
The first net is worse than the second
weights/ v sai-0.17 ( 75 games)
              wins        black       white
weights/   25 33.33%   13 33.33%   12 33.33%
sai-0.17   50 66.67%   26 66.67%   24 66.67%
                       39 52.00%   36 48.00%

Recent SAI 26x are much better!