search

NVIDIA / framework-reproducibility

Providing reproducibility in deep learning frameworks
Apache License 2.0
423 stars 40 forks source link

Will this package work on tf.keras model? #13

Closed leocnj closed 4 years ago

leocnj commented 4 years ago

Regarding "stock tensorflow 2.1", I installed tensorflow by using pip install tensorflow-gpu==2.1.0, shall I need strictly follow the pip installation command in your README file?

Also, if using tf.keras, shall I expect to get deterministic experiment results?

duncanriach commented 4 years ago

No, pip install tensorflow-gpu==2.1.0 and pip install tensorflow=2.1.0 do the same thing. With TensorFlow 2.1 and onwards, tensorflow-gpu and tensorflow are essentially the same package. I chose to show the newer way of doing things in my README file.

Yes, using tf.keras, you should get all the deterministic goodness that is now in stock TensorFlow. Remember that there are some TensorFlow ops, which may be exposed through Keras, that will inject non-determinism, so keep any eye out for that.

Use the following function to get and print a summary of your model weights at the end of training. If the summary is the same on separate training runs then your model is training with perfect reproducibility.

def summarize_weights(model):
  w = model.get_weights()
  return sum(map(lambda x: x.sum(), w))

Note: I intend to add this function to the tensorflow-determinism package at some point.