search

guitanj / AutoPK

Try to draw the elo-playouts curve of those leelazero networks
GNU General Public License v3.0
11 stars 4 forks source link

Is the LeelaZero LCBN2 version available? #1

Open Weberkx3 opened 5 years ago

Weberkx3 commented 5 years ago

The LCBN2 version is obviously powerful, and it works better at low Playout. Can it be included in the test?

image

https://cdn.discordapp.com/attachments/417022162348802050/560100473156403201/leelaz.exe

guitanj commented 5 years ago

sorry,I have no idea of this leelazero LCBN2 version,and I just want to find how strong these weights are,not engine. It's hard to find which is the best engine.So many parameters can take effect...

Weberkx3 commented 5 years ago

image

LCBN2 has been implemented as the new LeelaZero 0.17 algorithm, LCBN2 is very powerful and affects the strength of each weight at different Playouts.

At low Playout, the intensity of LCBN2 increases more, and the high Playout will approach the original strength, but more tests are needed.

guitanj commented 5 years ago

Thank you for explain! I have noticed that lz0.17 return lcb winrate,but still decide with normal winrate,Are there some parameters can change this?

Weberkx3 commented 5 years ago

I don't know, but LeelaZero 0.17 is theoretically stronger than LeelaZero 0.16 and does not require parameters.

Your test is the most complete I have ever seen. If you will be using LeelaZero 0.17, I am looking forward to seeing the results.

I also want to help with the test, but how to assign the game is a problem.

guitanj commented 5 years ago

Can you speak Chinese? I released executions-win64 version yesterday. If there is no parameter to use lcbrate then I have to change the program to use it.

guitanj commented 5 years ago

I don't know, but LeelaZero 0.17 is theoretically stronger than LeelaZero 0.16 and does not require parameters.

Your test is the most complete I have ever seen. If you will be using LeelaZero 0.17, I am looking forward to seeing the results.

I also want to help with the test, but how to assign the game is a problem.

Recently,I did some test and discuss the lcb with many FANS on Internet. Your are right i think,LCB is powerful with low playouts（above 20 playouts, beyond 6400po） And there is no parameters to effect LCB,because lz 0.17 changed the dicision mode,LCB‘s winrarate now replaced visits.

Weberkx3 commented 5 years ago

那就中文溝通吧，LCB已經是0.17預設的模式 我是想協助測試，但是不知道該怎麼幫忙

我有一張RTX2080

guitanj commented 5 years ago

太好了🤣 是的，通過這幾天的測試，我看到lz0.17已經是優先通過勝率選點了。在gtp命令genmove的結果返回中，有通過（V：）和（lcb：）返回的兩個勝率，目前觀察是lcb的值始終低於V值。 在低於20po時，我還觀察到lcb值會出現輸出為零的現象，所以我在上面的回覆中說要高於20po，lcb機制才會有效。 我用我自己編寫的測試程序進行了交換測試，方法是用lz0.17引擎和20190219發佈的老的Next引擎（還是優先visits數選點），交換黑白各測100局。 对比一下： （600po這個沒有用上面的方法測試，不過應該也可以說明問題） 220.gz B-Next-600po vs ELFV0 W-LZ0.17-600po 48:52 220.gz B-LZ0.17-600po vs ELFV0 W-LZ0.17-600po 73:27 lcb对40b低po的提升还是比较大的

但是在6400po,感觉就不明显了： ELFV2.gz B-LZ17-6400po vs ELFV2.gz W-Next-6400po 42:58 ELFV2.gz B-Next-6400po vs ELFV2.gz W-LZ0.17-6400po 41:59

我還測試了20po，同157權重，結果也是很明顯的lz0.17提升棋力。 關於我的測試程序，我目前發佈了一個release版本，裡面有兩個zip包，分別是同po測試和同時間測試，您可以下載先用用看，完善測試案例。使用方法我在發佈的頁面也做了簡單說明，有問題可以與我聯絡qq：七一二四五二四六

Weberkx3 commented 5 years ago

image 順利運作了，想問問怎麼生成ELO表格 我運算出的數據要怎麼做下一步處理?

guitanj commented 5 years ago

根據ELO的計算公式，比如A權重對戰B權重，勝率為Winrate，在excel中，可以這樣寫：=B權重的ELO值+(400*LOG(1/(1-Winrate)-1))。 我通過v0自戰，生成了elfv0的ELO值（指定2po的ELO值為2000），這樣就可以逐步計算生成其他權重的ELO值了。

guitanj commented 5 years ago

image 順利運作了，想問問怎麼生成ELO表格 我運算出的數據要怎麼做下一步處理?

更新了2.0程序，在原有功能基础上： 增加了图形界面，可导入棋谱指定开局进行批量测试 实时绘制胜率曲线，并在对局结束后保存

Weberkx3 commented 5 years ago

image 續關功能我做好了，原理是檔名核對

image SGF紀錄細節也做好了

不知道該怎麼合併，如果你有比較穩定的版本，我們再互相合併版本看看

guitanj commented 5 years ago

sorry，我不明白這兩個功能是什麼意思。。。 續關 是 中斷後繼續的功能嗎 sgf記錄細節，程序里有的，目前是選擇性的記錄信息。

昨晚更新了一個圖形界面的v2.1版本。

Weberkx3 commented 5 years ago

image

原理是在 startPK裡面，搜尋勝利與失敗的棋譜檔名，首先，每一個棋譜都有編號，也有特定的playout數字，所以棋譜檔名是不重複的。

因為勝利或失敗，會被儲存成不同名稱，直接比對就可以。

我的技術不太好，code改了很多，可能不太好合併

guitanj commented 5 years ago

你是根據對戰結果的sgf文件名統計勝負情況么。。。這個不需要這樣統計啊，我在結果記錄文件（PKResult.txt或者PKResult-ST.txt）中已經保存了相關信息了。

Weberkx3 commented 5 years ago

startPK會回傳勝負結果到最底下進行統計，紀錄在PKResult.txt

我只是在startPK裡面，額外的新增一個勝負統計方式 如果存在sgf檔案，就表示這場已經打過了 就跳過啟動goEngine的各種步驟，直接回傳勝負結果

image

目前我在研究多個權重，自動循環比賽的部分 修改的code超過400行，希望沒有BUG... @@"

這是我的版本： https://drive.google.com/open?id=1KmKpZYoJ3Rbp6zgsnTGUxOiNSZMTFB8c

我的config改成這樣： [HowToFight] dirName = LZ_223_FullTest start = 1 end = 100 pklist = 2,2;25,25;100,100;400,400

[Engine_Master] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_223_c9fb22c7.gz name = LZ_223

[Engine_01] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_220_565f8182.gz name = LZ_220

[Engine_02] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_210_1e6c31cc.gz name = LZ_210

[Engine_03] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_200_50b98433.gz name = LZ_200

[Engine_04] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_190_ef09cd53.gz name = LZ_190

[Engine_05] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = LZ_180_da3d663a.gz name = LZ_180

[Engine_06] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = ELFv2_05dbca15.gz name = ELFv2

[Engine_07] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = ELFv0_62b5417b.gz name = ELFv0

[Engine_08] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = 15b_204_0a963117.gz name = 15b_204

[Engine_09] enginedir = ../_e/ engine = leelaz.exe -g -t1 --batchsize 1 -r5 --noponder weightdir = ../_w/ weight = 15b_157_d351f06e.gz name = 15b_157

guitanj commented 5 years ago

很有意思的一個獨特的版本，獨立保持版本比較好啦。