search

keras-team / keras-io

Keras documentation, hosted live at keras.io
Apache License 2.0
2.69k stars 2.01k forks source link

GTP miniature model don't load. #1884

Open alkaou opened 1 week ago

alkaou commented 1 week ago

Issue Type

Others

Source

source

Keras Version

keras 3

Custom Code

Yes

OS Platform and Distribution

Windows 11

Python version

3.10

GPU model and memory

CPU

Current Behavior?

I looked at the miniature GPT code in the documentation to do my project. Everything was working fine until I registered my model after training. Now when I try to load my model to use it, I get an error of "ustom_objects". I checked paI haven't found a solution.

Standalone code to reproduce the issue or tutorial link

when I run the code to load the model

from tensorflow.keras import models
model = models.load_model("model.keras")

I'm getting the error

Relevant log output

I'm getting this error:

Python Projects\ia_code\venv\lib\site-packages\keras\src\layers\layer.py:361: UserWarning: `build()` was called on layer 'token_and_position_embedding_13', however the layer does not have a `build()` method implemented and it looks like it has unbuilt state. This will cause the layer to be marked as built, despite not being actually built, which may cause failures down the line. Make sure to implement a proper `build()` method.
  warnings.warn(
--------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\ops\operation.py:208, in Operation.from_config(cls, config)
    207 try:
--> 208     return cls(**config)
    209 except Exception as e:

TypeError: TransformerBlock.__init__() got an unexpected keyword argument 'trainable'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:718, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    717 try:
--> 718     instance = cls.from_config(inner_config)
    719 except TypeError as e:

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\ops\operation.py:210, in Operation.from_config(cls, config)
    209 except Exception as e:
--> 210     raise TypeError(
    211         f"Error when deserializing class '{cls.__name__}' using "
    212         f"config={config}.\n\nException encountered: {e}"
    213     )

TypeError: Error when deserializing class 'TransformerBlock' using config={'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}.

Exception encountered: TransformerBlock.__init__() got an unexpected keyword argument 'trainable'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:718, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    717 try:
--> 718     instance = cls.from_config(inner_config)
    719 except TypeError as e:

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\models\model.py:517, in Model.from_config(cls, config, custom_objects)
    515     from keras.src.models.functional import functional_from_config
--> 517     return functional_from_config(
    518         cls, config, custom_objects=custom_objects
    519     )
    521 # Either the model has a custom __init__, or the config
    522 # does not contain all the information necessary to
    523 # revive a Functional model. This happens when the user creates
   (...)
    526 # In this case, we fall back to provide all config into the
    527 # constructor of the class.

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\models\functional.py:517, in functional_from_config(cls, config, custom_objects)
    516 for layer_data in config["layers"]:
--> 517     process_layer(layer_data)
    519 # Then we process nodes in order of layer depth.
    520 # Nodes that cannot yet be processed (if the inbound node
    521 # does not yet exist) are re-enqueued, and the process
    522 # is repeated until all nodes are processed.

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\models\functional.py:501, in functional_from_config.<locals>.process_layer(layer_data)
    500 else:
--> 501     layer = serialization_lib.deserialize_keras_object(
    502         layer_data, custom_objects=custom_objects
    503     )
    504 created_layers[layer_name] = layer

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:720, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    719 except TypeError as e:
--> 720     raise TypeError(
    721         f"{cls} could not be deserialized properly. Please"
    722         " ensure that components that are Python object"
    723         " instances (layers, models, etc.) returned by"
    724         " `get_config()` are explicitly deserialized in the"
    725         " model's `from_config()` method."
    726         f"\n\nconfig={config}.\n\nException encountered: {e}"
    727     )
    728 build_config = config.get("build_config", None)

TypeError: <class '__main__.TransformerBlock'> could not be deserialized properly. Please ensure that components that are Python object instances (layers, models, etc.) returned by `get_config()` are explicitly deserialized in the model's `from_config()` method.

config={'module': None, 'class_name': 'TransformerBlock', 'config': {'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}, 'registered_name': 'TransformerBlock', 'build_config': {'input_shape': [None, 80, 256]}, 'name': 'transformer_block_3', 'inbound_nodes': [{'args': [{'class_name': '__keras_tensor__', 'config': {'shape': [None, 80, 256], 'dtype': 'float32', 'keras_history': ['token_and_position_embedding_11', 0, 0]}}], 'kwargs': {}}]}.

Exception encountered: Error when deserializing class 'TransformerBlock' using config={'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}.

Exception encountered: TransformerBlock.__init__() got an unexpected keyword argument 'trainable'

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
Cell In[83], line 5
      1 # from tensorflow.keras.utils import custom_object_scope
      2 # from tensorflow.keras.utils import get_custom_objects
      3 from tensorflow.keras import models
----> 5 br_model = models.load_model("E:\\Alkaou\Python Projects\\models\\br_model.keras")
      7 br_model.summary()

File E:\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_api.py:176, in load_model(filepath, custom_objects, compile, safe_mode)
    173         is_keras_zip = True
    175 if is_keras_zip:
--> 176     return saving_lib.load_model(
    177         filepath,
    178         custom_objects=custom_objects,
    179         compile=compile,
    180         safe_mode=safe_mode,
    181     )
    182 if str(filepath).endswith((".h5", ".hdf5")):
    183     return legacy_h5_format.load_model_from_hdf5(
    184         filepath, custom_objects=custom_objects, compile=compile
    185     )

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_lib.py:152, in load_model(filepath, custom_objects, compile, safe_mode)
    147     raise ValueError(
    148         "Invalid filename: expected a `.keras` extension. "
    149         f"Received: filepath={filepath}"
    150     )
    151 with open(filepath, "rb") as f:
--> 152     return _load_model_from_fileobj(
    153         f, custom_objects, compile, safe_mode
    154     )

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_lib.py:170, in _load_model_from_fileobj(fileobj, custom_objects, compile, safe_mode)
    168 # Construct the model from the configuration file in the archive.
    169 with ObjectSharingScope():
--> 170     model = deserialize_keras_object(
    171         config_dict, custom_objects, safe_mode=safe_mode
    172     )
    174 all_filenames = zf.namelist()
    175 if _VARS_FNAME + ".h5" in all_filenames:

File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:720, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
    718     instance = cls.from_config(inner_config)
    719 except TypeError as e:
--> 720     raise TypeError(
    721         f"{cls} could not be deserialized properly. Please"
    722         " ensure that components that are Python object"
    723         " instances (layers, models, etc.) returned by"
    724         " `get_config()` are explicitly deserialized in the"
    725         " model's `from_config()` method."
    726         f"\n\nconfig={config}.\n\nException encountered: {e}"
    727     )
    728 build_config = config.get("build_config", None)
    729 if build_config and not instance.built:

TypeError: <class 'keras.src.models.functional.Functional'> could not be deserialized properly. Please ensure that components that are Python object instances (layers, models, etc.) returned by `get_config()` are explicitly deserialized in the model's `from_config()` method.

config={'module': 'keras.src.models.functional', 'class_name': 'Functional', 'config': {'name': 'functional_11', 'trainable': True, 'layers': [{'module': 'keras.layers', 'class_name': 'InputLayer', 'config': {'batch_shape': [None, 80], 'dtype': 'int32', 'sparse': False, 'name': 'input_layer_8'}, 'registered_name': None, 'name': 'input_layer_8', 'inbound_nodes': []}, {'module': None, 'class_name': 'TokenAndPositionEmbedding', 'config': {'maxlen': 80, 'vocab_size': 20000, 'embed_dim': 256, 'trainable': True, 'dtype': 'float32'}, 'registered_name': 'TokenAndPositionEmbedding', 'build_config': {'input_shape': [None, 80]}, 'name': 'token_and_position_embedding_11', 'inbound_nodes': [{'args': [{'class_name': '__keras_tensor__', 'config': {'shape': [None, 80], 'dtype': 'int32', 'keras_history': ['input_layer_8', 0, 0]}}], 'kwargs': {}}]}, {'module': None, 'class_name': 'TransformerBlock', 'config': {'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}, 'registered_name': 'TransformerBlock', 'build_config': {'input_shape': [None, 80, 256]}, 'name': 'transformer_block_3', 'inbound_nodes': [{'args': [{'class_name': '__keras_tensor__', 'config': {'shape': [None, 80, 256], 'dtype': 'float32', 'keras_history': ['token_and_position_embedding_11', 0, 0]}}], 'kwargs': {}}]}, {'module': 'keras.layers', 'class_name': 'Dense', 'config': {'name': 'dense_11', 'trainable': True, 'dtype': 'float32', 'units': 20000, 'activation': 'linear', 'use_bias': True, 'kernel_initializer': {'module': 'keras.initializers', 'class_name': 'GlorotUniform', 'config': {'seed': None}, 'registered_name': None}, 'bias_initializer': {'module': 'keras.initializers', 'class_name': 'Zeros', 'config': {}, 'registered_name': None}, 'kernel_regularizer': None, 'bias_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None}, 'registered_name': None, 'build_config': {'input_shape': [None, 80, 256]}, 'name': 'dense_11', 'inbound_nodes': [{'args': [{'class_name': '__keras_tensor__', 'config': {'shape': [None, 80, 256], 'dtype': 'float32', 'keras_history': ['transformer_block_3', 0, 0]}}], 'kwargs': {}}]}], 'input_layers': [['input_layer_8', 0, 0]], 'output_layers': [['dense_11', 0, 0], ['transformer_block_3', 0, 0]]}, 'registered_name': 'Functional', 'build_config': {'input_shape': None}, 'compile_config': {'optimizer': 'Adam', 'loss': [{'module': 'keras.losses', 'class_name': 'SparseCategoricalCrossentropy', 'config': {'name': 'sparse_categorical_crossentropy', 'reduction': 'sum_over_batch_size', 'from_logits': True, 'ignore_class': None}, 'registered_name': None}, None], 'loss_weights': None, 'metrics': None, 'weighted_metrics': None, 'run_eagerly': False, 'steps_per_execution': 1, 'jit_compile': False}}.

Exception encountered: <class '__main__.TransformerBlock'> could not be deserialized properly. Please ensure that components that are Python object instances (layers, models, etc.) returned by `get_config()` are explicitly deserialized in the model's `from_config()` method.

config={'module': None, 'class_name': 'TransformerBlock', 'config': {'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}, 'registered_name': 'TransformerBlock', 'build_config': {'input_shape': [None, 80, 256]}, 'name': 'transformer_block_3', 'inbound_nodes': [{'args': [{'class_name': '__keras_tensor__', 'config': {'shape': [None, 80, 256], 'dtype': 'float32', 'keras_history': ['token_and_position_embedding_11', 0, 0]}}], 'kwargs': {}}]}.

Exception encountered: Error when deserializing class 'TransformerBlock' using config={'embed_dim': 256, 'num_heads': 2, 'ff_dim': 256, 'trainable': True, 'dtype': 'float32'}.

Exception encountered: TransformerBlock.__init__() got an unexpected keyword argument 'trainable'
sachinprasadhs commented 1 week ago

Hi, If you are referring to the tutorial here https://keras.io/examples/generative/text_generation_with_miniature_gpt/, could you please let us know if you are modifying anything. Thanks

alkaou commented 6 days ago

Hi, If you are referring to the tutorial here https://keras.io/examples/generative/text_generation_with_miniature_gpt/, could you please let us know if you are modifying anything. Thanks

I didn't change anything in the code. I made a git repository. you can please visit my code to help me.

code is here : https://github.com/alkaou/GenIA_LLM.git

sachinprasadhs commented 3 days ago

I think the issue is solved with the comment here to include **kwargs in the custom class https://github.com/keras-team/keras/issues/19899#issuecomment-2185256214. You can close the issue. Thanks