Setting the seed. Seed can be None but if given is a integer and is given in the tune config. That seed will initialize python, numpy and torch seeds in the TuneWrapper class init. Then numbers will be drown randomly (always the same if the seed is set) and those values will be the seeds for the trials inside tune. So each tune trial/experiment will have is own seed set in a reproducible manner because is dependant on the overall user given seed.
Seed determine also weights initialization for the specific model (1 model per trial). Model are initialized always to same parameters if seed is given.
Added debug mode to explroe weights initialization, seeds and raw output predicgtion on validation set for best model. All this files are outputed only if debug_mode is activated. They will be present in the debug dir under the TuneRun results subdir. This files are used for understanding reproduciobility across identical runs of tune.
handle_tune nf-test has been created and dnatofloat on cpu is set as proxy for reproducibility. More work here in the future
Now tune configs can specify a run_params key that handles RunConfig auxiliary information like stop criteria necessary for the FIFOscheduler now present in the dnatofloat cpu tune config.
TODO for the future is decide wich flavor of the nf-test tune should be put as guthub action to test reproducibility.
Changes aported:
TODO for the future is decide wich flavor of the nf-test tune should be put as guthub action to test reproducibility.