Dboingue
commented
3 years ago - [ ] complement the above with new information about one paragraph that could relate to the master network ML setup definition.
- [ ] edit the prose for more efficient style
- [ ] find the link to source code introduction with mention to big master network, without its ML context, in favor of the actual NNue instances used at end of NNue sub-pipeline.
- [ ] maybe move this or make a referring issue in stockfisch.,
- [ ] link to tangential issue about documenting the whole of stockfish parameters set definition and evolution (focus of SF11)
- [ ] find a way to make this quest for clues toward building a faithful high level algorithm or mathematical description
for some that is redundant, an algorithm is already a mathematical object, but I could distinguish algorithm and implementation of algorithm. or if the implementation in source code is the algorithm, then I would need a high level representation).
I am trying to fill in some gaps in the documentation (including source code comments),and I have seen in that file change or addition intro blurb with refereences to transformer smaller nets links.
The data used for the master to transformer developmental smaller nets (my undestanding) is describedthere. But I may have missed the equivalent for the master net. From a ML point of view, the transformer explanation was understandable: in short: using the master output on common positions input as SL targets in some global loss function (some use the oracle word).
And I just started reading this repo mains page about data generation, using the classical evalution.
I find this diversity of neural net training method documentation a bit confusing. Are you all using the same terminology.
Has master network (too big to fit in hardware and time constraints of engine competitions I suppose), changes designation. Can anyone help me fill in the gaps, or documentation discrepancies to get up to date toward getting the full big ML pictures of the master net training setup. and clarify, when here the words "generating training data" is using, which it might fit in the master to transformer scheme. Related, is how in this training data related to the SF14 NNue training process (does it still use a big network with the same training setup which I still don't know (repeating mysefl) as described in the SF12 NNue intro source code or github repo of it). I will find the link for completion in case it not clear which i mean). days.
Thanks you for pointers. This issue i have also posted in aother NNue training related repo. I am not sure to understand the division of objectives and which repo does what in relation to the officel SF with NNue. I understand that they are modular and that is good for mutliplying experiments with other AB engines trying to jump on the train. but I would like to at least get the SF14 NNue training setup whereabouts... I am not programmer enough and I don't need to learn about the opmitization of NN, or NNue improvements in implementations. The science of the training should still be describable in Machine learning terms with input and output well defined (i couls help once i gather the info without all the distractions to filter out). Specifically RL or SL, target definitions. database construction. in RL exploration verus "exploitation" scheduling. or if any self-play at all using the same random initial exploration as a0 or lc 0. where is the master net self play data.
All my qeustions could be documented separaterly from the other challenges solved with NNue implementations.
I hope i surrounded enough the nature of my search. it is an eternal challenge. documentation above source code.
help! i am patient. pointers and hint however small can help.