search

lightvector / KataGo

GTP engine and self-play learning in Go
https://katagotraining.org/
Other
3.56k stars 564 forks source link

Data lost in one katago network training is applied in the next net? #545

Open jaewoo1120 opened 3 years ago

jaewoo1120 commented 3 years ago

Even when a new network emerges, data from previous network training is uploaded. Will these data be applied when creating the next network?

Friday9i commented 3 years ago

Yes More specifically, training a new network make use of the newest available data (ie selfplay with the latest network) AND a small percentage of all the data from the last weeks/months: all these data are shuffled and used for training. Why doing that? Many reasons: mixing data from many nets ensures more diversity, it limits the risks of overfitting, if a net is bad then its data have a limited impact, the volume of data needed for training is high (hence, without using data from old nets, we would need to wait several days / weeks of selfplay before training a new net), etc...

jaewoo1120 commented 3 years ago

Thanks for the reply. I'm glad the data wasn't completely discarded. :+1: