Setting the random seed

[ahwillia]

Thanks for writing a great and user-friendly package.

Sorry if I missed it, but I couldn't find any examples showing how to best set and store a unique random seed for each model. My use case is that I want to train multiple networks with identical hyperparameters but different random weights and training examples. Are there tools to help facilitate and keep track of this or should I just use tf.random.set_seed(...) and np.random.seed(...)?

[syncrostone]

Ah, great question! I should add something to the docs about this.

Yes, using tf.random.set_seed, np.random.seed, and random.seed should do the trick.

E.g.

import tensorflow as tf
import numpy as np
import random

def reset_seeds(seed):
    tf.compat.v2.random.set_seed(seed) # This version works with both tf1 and tf2
    random.seed(seed)
    np.random.seed(seed)

[ahwillia]

Thanks for the quick response! It might also be nice to a parameter like init_seed to the model RNN class which automatically sets the seeds before initialization, and then could be saved alongside the (trained) weights in a metadata file?

[syncrostone]

A parameter like init_seed could be added in the model RNN class / the initialization class, but I think it would be misleading to then save this alongside the trained weights because weights can be trained on multiple occasions after initializations, and if the user calls other operations between the initialization and training, the init_seed would not fully determine the trained weights.

For init_seed, I would expect the user to pass in the seed to be used with all the above functions and so be able to keep track of it on their end. Were you thinking of just having a reset_seeds boolean argument with the initialization randomly choosing a seed to reset to?

[ahwillia]

Yeah it could be cleanest to just let the user keep track of these things themselves. I personally wouldn't retrain the same model instance, but different people might like to organize their hyperparameter sweeps in different ways. Thanks!