OmnipotentEntity
commented
1 year ago From lightvector earlier today on the Discord
Q (from me):
Is 1.12.0 going to be the new models update? If so, is self-play training going to move over to b18 immediately or is it going to wait a few weeks/months?
A (from lightvector):
Yes, that's the plan for 1.12.0.
I think self-play training will probably move a few weeks after, to give people time to upgrade, and/or to give time in case broader issues are reported with using the new architecture once many more people are trying it. I think I want to have b18 driving the learning since it learns faster, but I'm not sure. We're going to be running a version that is a bit higher LR and weaker than the 60 block net, so at least nominally it's a weakening of the data quality unless we also boost visits. (Then again, when changing from 40 block data to 60 block data we didn't see a huge boost in improvement rate, so maybe it doesn't matter).
Actually I'm not really sure what configuration to go with. Please feel free to weigh in! 😋
There's also a b28 or b32 new architecture net that I'll be starting training on at some point too, once I have spare GPUs, to get a bigger version of the b18. (This will become possible if we can stop training at least one of 40b and 60b).
He seemed eager to get 1.12.0 out of the door. But he has some things he wanted to research first (regarding positions with cyclic groups).
gaoruihua111
ohminzhaogo
Will you update source in KataGo v1.12.0 that can train new improved neural net architectures?