Keras load LSTM/GRU model with constant mask/initial_state will raise error

[RunnerZhong]

Issue Type Bug

Source binary

Tensorflow Version TF 2.6.3

Custom Code Yes

OS Platform and Distribution RedHat 7

Mobile device No response

Python version 3.8

Bazel version No response

GCC/Compiler version No response

CUDA/cuDNN version No response

GPU model and memory No response

Current Behaviour?

Traceback (most recent call last): File "/home/runner/work/sample_code/example.py", line 28, in dd = tf.keras.models.load_model("./lstm.h5", compile=False, options=load_options) File "/home/runner/anaconda3/envs/latest_env/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/runner/anaconda3/envs/latest_env/lib/python3.8/site-packages/keras/backend.py", line 1470, in int_shape shape = x.shape AttributeError: 'float' object has no attribute 'shape'

Standalone code to reproduce the issue

import numpy as np import tensorflow as tf import tensorflow.keras.backend as K

input_t = tf.keras.Input(shape=(28, 10), batch_size=2, dtype="float32")

rdm_value = np.ones([2, 28]).astype(np.float32) rdm_value[:, 20:] = 0 mask_value = K.constant(np.array(rdm_value), dtype='bool')

m_state = tf.keras.initializers.GlorotUniform()(shape=[2, 6], dtype='float32') c_state = tf.keras.initializers.GlorotUniform()(shape=[2, 6], dtype='float32') init_state_value = [m_state, c_state]

lstm = tf.keras.layers.LSTM(6, return_sequences=True, return_state=True, bias_initializer='random_uniform', time_major=False)(inputs=input_t, mask=mask_value, training=False, initial_state=init_state_value) keras_model = tf.keras.Model([input_t], lstm) keras_model.save('./lstm.h5')

load_options = tf.saved_model.LoadOptions(allow_partial_checkpoint=True) dd = tf.keras.models.load_model("./lstm.h5", compile=False, options=load_options) print(dd.inputs)

[sushreebarsa]

@RunnerZhong Could you have a look at the gist here and confirm the issue? Thank you!

[RunnerZhong]

yes, this is the issue that I met

[sachinprasadhs]

Could you try compiling the model and train it with some data to check if you are facing the same behavior as saving and loading non compiled model. Now, as the warning suggested, the compiled metrics has not been built yet, this will be empty till you train or evaluate the model. Additionally, if you want to go ahead with the non compiled model, you could try saving the weights and load the weights like below.

keras_model.save_weights('./lstm.h5')
keras_model.load_weights('./lstm.h5')

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[RunnerZhong]

I added compiling and train before save the model, but this issue still can reproduce.

[RunnerZhong]

[RunnerZhong]

You can have a try too, thanks your reply~

[sachinprasadhs]

Could you please provide the reproducible code which you have used to compile the model with sample input. Thanks!

[RunnerZhong]

`import numpy as np import tensorflow as tf import tensorflow.keras.backend as K

input_t = tf.keras.Input(shape=(28, 10), batch_size=2, dtype="float32")

rdm_value = np.ones([2, 28]).astype(np.float32) rdm_value[:, 20:] = 0 mask_value = K.constant(np.array(rdm_value), dtype='bool')

m_state = tf.keras.initializers.GlorotUniform()(shape=[2, 6], dtype='float32') c_state = tf.keras.initializers.GlorotUniform()(shape=[2, 6], dtype='float32') init_state_value = [m_state, c_state]

lstm = tf.keras.layers.LSTM(6, return_sequences=True, return_state=True, bias_initializer='random_uniform', time_major=False)(inputs=input_t, mask=mask_value, training=True, initial_state=init_state_value) keras_model = tf.keras.Model([input_t], lstm) keras_model.compile(optimizer="adam", loss="mean_squared_error") test_input = np.random.random(input_t.shape) test_target = [np.random.random((2, 28, 6)), np.random.random((2, 6)) ,np.random.random((2, 6))] keras_model.fit(test_input, test_target) keras_model.save('./lstm.h5')

load_options = tf.saved_model.LoadOptions(allow_partial_checkpoint=True) dd = tf.keras.models.load_model("./lstm.h5", compile=True, options=load_options) print(dd.inputs)`

[RunnerZhong]

You can reproduce issue with above sample, thanks.

[sachinprasadhs]

I was able to reproduce the behavior using Tf-Nightly(2.13), please find the Gist here. Thanks!

[grasskin]

@nkovela1