The hyper-parameter optimization will create a folder structure parameter. If the hyper-params are spectral_radius, and weight_init it will look like this:
After the untrainedmodel.pth has been created all following steps are deterministic as the only source of randomness is the creation of the model matrices.
The train_data.nc of the run folders will contain the used inputs/labels and created states which can now be generated.
Next the output layer can be optimized. For some (level-two) hyper-parameters the optimization of the states that are created by a given input sequence are invariant and can be reused. This is reflected in another sub directory of the folder structure:
These subdirectories will contain pinv and tik folders. They contain the now trained models and the corresponding predictions. Additionally, the params.json in which only the parameters defined in the parent paths are updated accordingly.
nmheim
commented
5 years ago
Trying to resolve #20
The hyper-parameter optimization will create a folder structure parameter. If the hyper-params are
spectral_radius, andweight_initit will look like this:After the untrained
model.pthhas been created all following steps are deterministic as the only source of randomness is the creation of the model matrices. Thetrain_data.ncof the run folders will contain the used inputs/labels and created states which can now be generated.Next the output layer can be optimized. For some (level-two) hyper-parameters the optimization of the states that are created by a given input sequence are invariant and can be reused. This is reflected in another sub directory of the folder structure:
These subdirectories will contain
pinvandtikfolders. They contain the now trained models and the corresponding predictions. Additionally, theparams.jsonin which only the parameters defined in the parent paths are updated accordingly.