Keras 3 with Pytorch backend ERROR - Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']

[sunilsarolkarcds]

We are developing Keras 3 model using Bidirectional LSTM using pytorch as a backend, we have trained the model and spent entire day training it and saved in the end in .keras extension, but we are not able to load back saved model with weights it gives error as below

/usr/local/lib/python3.10/dist-packages/keras/src/saving/saving_lib.py:418: UserWarning: Skipping variable loading for optimizer 'adam', because it has 44 variables whereas the saved optimizer has 32 variables. 
  trackable.load_own_variables(weights_store.get(inner_path))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-23-b0bf4265eb34>](https://localhost:8080/#) in <cell line: 3>()
      1 model.save('/content/model.keras')
      2 
----> 3 model = keras.models.load_model('/content/model.keras')

3 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/saving/saving_lib.py](https://localhost:8080/#) in _raise_loading_failure(error_msgs, warn_only)
    293         warnings.warn(msg)
    294     else:
--> 295         raise ValueError(msg)
    296 
    297 

ValueError: A total of 2 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:

Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']

List of objects that could not be loaded:
[<LSTMCell name=lstm_cell, built=True>, <LSTMCell name=lstm_cell, built=True>]

Pytorch and keras versions

import torch
print('torch.__version__',torch.__version__)
import os
os.environ["KERAS_BACKEND"] = "torch"
import keras
print('keras.__version__',keras.__version__)

OUTPUT

torch.__version__ 2.3.0+cu121
keras.__version__ 3.3.2

Sample model code

import numpy as np
import keras
from keras import layers

max_features = 20000  # Only consider the top 20k words
maxlen = 200  # Only consider the first 200 words of each movie review

os.environ["KERAS_BACKEND"] = "torch"
import keras
print(keras.__version__)

# Input for variable-length sequences of integers
inputs = keras.Input(shape=(None,), dtype="int32")
# Embed each integer in a 128-dimensional vector
x = layers.Embedding(max_features, 128)(inputs)
# Add 2 bidirectional LSTMs
x = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(x)
x = layers.Bidirectional(layers.LSTM(64))(x)
# Add a classifier
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs, outputs)
model.summary()

(x_train, y_train), (x_val, y_val) = keras.datasets.imdb.load_data(
    num_words=max_features
)
print(len(x_train), "Training sequences")
print(len(x_val), "Validation sequences")
# Use pad_sequence to standardize sequence length:
# this will truncate sequences longer than 200 words and zero-pad sequences shorter than 200 words.
x_train = keras.utils.pad_sequences(x_train, maxlen=maxlen)
x_val = keras.utils.pad_sequences(x_val, maxlen=maxlen)

Model training

model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(x_train[:100], y_train[:100], batch_size=32, epochs=1, validation_data=(x_val[:20], y_val[:20]))

Saving and loading back the model

model.save('/content/model.keras')

model = keras.models.load_model('/content/model.keras')

OUTPUT

/usr/local/lib/python3.10/dist-packages/keras/src/saving/saving_lib.py:418: UserWarning: Skipping variable loading for optimizer 'adam', because it has 44 variables whereas the saved optimizer has 32 variables. 
  trackable.load_own_variables(weights_store.get(inner_path))
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-23-b0bf4265eb34>](https://localhost:8080/#) in <cell line: 3>()
      1 model.save('/content/model.keras')
      2 
----> 3 model = keras.models.load_model('/content/model.keras')

3 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/saving/saving_lib.py](https://localhost:8080/#) in _raise_loading_failure(error_msgs, warn_only)
    293         warnings.warn(msg)
    294     else:
--> 295         raise ValueError(msg)
    296 
    297 

ValueError: A total of 2 objects could not be loaded. Example error message for object <LSTMCell name=lstm_cell, built=True>:

Layer 'lstm_cell' expected 3 variables, but received 0 variables during loading. Expected: ['kernel', 'recurrent_kernel', 'bias']

List of objects that could not be loaded:
[<LSTMCell name=lstm_cell, built=True>, <LSTMCell name=lstm_cell, built=True>]

Can someone help on this, is there any workaround to load the existing saved model

[fchollet]

That's a pretty interesting bug, thanks for the report. Deserializing a model that contains Bidirectional(LSTM) seems to leave the wrapped layer unbuilt which causes weight loading to fail. We'll fix that.

As a workaround, just call new_model.load_weights("model.keras") after re-instanting your model (new_model) from code.

[fchollet]

This is now fixed at HEAD.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[sachinprasadhs]

Issue seems to still persist with tensorflow backend using keras-nightly, https://gist.github.com/sachinprasadhs/eb2d2af87610b06cad3843ac5699089a

[normanlmfung]

Hi team, I just reinstalled Keras, now version is : keras-3.4.1, still having same trouble.