search

CuriosAI / sai

SAI: a fork of Leela Zero with variable komi.
GNU General Public License v3.0
103 stars 11 forks source link

what is the meaning of the experiment type of matchs #131

Closed l1t1 closed 3 years ago

l1t1 commented 3 years ago

and how the games of each type be decided?

Vandertic commented 3 years ago

It's explained in the Telegram channel (link in SAI homepage). Current experiment are networks trained with KLE loss (cross-entropy) instead of MSE loss (difference squared) for the value head. Four nets are trained for each generation, starting from the previous best KLE network. (We make a small number of matches with low visits to get a reasonable evaluation of strength.) The experiment will be evaluated after 10 generations, and we are at 7. Previously we had a huge factorial experiment in training 20x256 networks at generation g1a8 (SAI424). Results were appalling though, because no matter what we changed, we could not get networks clearly stronger than the same generation's 12x256.

l1t1 commented 3 years ago

thanks, the Telegram channel is banned in my area

Vandertic commented 3 years ago

Oh, I am sorry about that. Maybe someone can find a way around to put the same content in a reachable place?

l1t1 commented 3 years ago

this net (779e82d7) seems very strong, could you promote it force?

779e82d7 VS 79cec3c4 18 : 2 : 5 (76.00%)
Vandertic commented 3 years ago

It is a KLE network. Surely it is strong, but its current Elo is overestimated because no subsequent network was tested against it.

Anyway we will be passing to KLE training in a few days. (Sorry, but I have to prepare a conference.)