search

keras-team / keras-tuner

A Hyperparameter Tuning Library for Keras
https://keras.io/keras_tuner/
Apache License 2.0
2.86k stars 397 forks source link

tuner.get_best_models() issues warning Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values #982

Open giulatona opened 11 months ago

giulatona commented 11 months ago

Describe the bug Calling tuner.get_best_models() at the end of hp tuning issues several warnings due to missing checkpoints.

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use status.expect_partial(). See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function. WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-0.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-0.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-1.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).layer_with_weights-1.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.1 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.2 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.3 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.4 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.5 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.6 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.7 WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).optimizer._variables.8

To Reproduce The official example in the documentation shows this problem. It can be reproduced in the related Colab Notebook.

Expected behavior No warnings should be issued.

Would you like to help us fix it? I do not know the codebase enough.

grmelacz commented 6 months ago

I might not be your case, however I have fixed this error by removing my custom-defined metric (in my case f1score) from the model used in build_model(). It seems it was not saved properly/not implemented properly for this specific (tuner) case.