alkaou
commented
5 days ago This Pull Request introduces a new example demonstrating the implementation of a Generative Pretrained Transformer (GPT) using Keras. The example showcases how to build, train, and generate text with a GPT model, providing a comprehensive guide for users interested in natural language processing (NLP) and deep learning.
After training, once saved keep the model, it can no longer be loaded.
Hist = model.fit(
train_data_generator,
epochs=Epochs,
steps_per_epoch=eval_interval,
validation_data=val_data_generator,
validation_steps=eval_iters,
)
model_name = "br_bigram_model.h5"
# save the model
model.save(model_name)
from keras.models import load_model
custom_objects = {
"FeedForward": FeedForward,
"Block": Block,
"BigramLanguageModel": BigramLanguageModel,
}# Load the model
bigram_model = load_model(model_name, custom_objects=custom_objects)
bigram_model .summary()Here is the error we always return:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:718, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
717 try:
--> 718 instance = cls.from_config(inner_config)
719 except TypeError as e:
Cell In[157], line 69, in BigramLanguageModel.from_config(cls, config)
67 @classmethod
68 def from_config(cls, config):
---> 69 return cls(**config)
TypeError: BigramLanguageModel.__init__() got an unexpected keyword argument 'name'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
Cell In[173], line 11
4 custom_objects = {
5 "FeedForward": FeedForward,
6 "Block": Block,
7 "BigramLanguageModel": BigramLanguageModel,
8 }
10 # Charger le modèle
---> 11 br_bigram_model = load_model(model_name, custom_objects=custom_objects)
12 br_bigram_model.summary()
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_api.py:176, in load_model(filepath, custom_objects, compile, safe_mode)
173 is_keras_zip = True
175 if is_keras_zip:
--> 176 return saving_lib.load_model(
177 filepath,
178 custom_objects=custom_objects,
179 compile=compile,
180 safe_mode=safe_mode,
181 )
182 if str(filepath).endswith((".h5", ".hdf5")):
183 return legacy_h5_format.load_model_from_hdf5(
184 filepath, custom_objects=custom_objects, compile=compile
185 )
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_lib.py:152, in load_model(filepath, custom_objects, compile, safe_mode)
147 raise ValueError(
148 "Invalid filename: expected a `.keras` extension. "
149 f"Received: filepath={filepath}"
150 )
151 with open(filepath, "rb") as f:
--> 152 return _load_model_from_fileobj(
153 f, custom_objects, compile, safe_mode
154 )
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\saving_lib.py:170, in _load_model_from_fileobj(fileobj, custom_objects, compile, safe_mode)
168 # Construct the model from the configuration file in the archive.
169 with ObjectSharingScope():
--> 170 model = deserialize_keras_object(
171 config_dict, custom_objects, safe_mode=safe_mode
172 )
174 all_filenames = zf.namelist()
175 if _VARS_FNAME + ".h5" in all_filenames:
File E:\Alkaou\Python Projects\ia_code\venv\lib\site-packages\keras\src\saving\serialization_lib.py:720, in deserialize_keras_object(config, custom_objects, safe_mode, **kwargs)
718 instance = cls.from_config(inner_config)
719 except TypeError as e:
--> 720 raise TypeError(
721 f"{cls} could not be deserialized properly. Please"
722 " ensure that components that are Python object"
723 " instances (layers, models, etc.) returned by"
724 " `get_config()` are explicitly deserialized in the"
725 " model's `from_config()` method."
726 f"\n\nconfig={config}.\n\nException encountered: {e}"
727 )
728 build_config = config.get("build_config", None)
729 if build_config and not instance.built:
TypeError: <class '__main__.BigramLanguageModel'> could not be deserialized properly. Please ensure that components that are Python object instances (layers, models, etc.) returned by `get_config()` are explicitly deserialized in the model's `from_config()` method.
config={'module': None, 'class_name': 'BigramLanguageModel', 'config': {'name': 'bigram_language_model_7', 'trainable': True, 'dtype': 'float32', 'vocab_size': 108, 'n_embd': 64, 'block_size': 32, 'n_head': 4, 'n_layer': 4}, 'registered_name': 'BigramLanguageModel', 'build_config': {'input_shape': [16, 32]}, 'compile_config': {'optimizer': {'module': 'keras.optimizers', 'class_name': 'Adam', 'config': {'name': 'adam', 'learning_rate': 0.0010000000474974513, 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'loss_scale_factor': None, 'gradient_accumulation_steps': None, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}, 'registered_name': None}, 'loss': None, 'loss_weights': None, 'metrics': None, 'weighted_metrics': None, 'run_eagerly': False, 'steps_per_execution': 1, 'jit_compile': False}}.
Exception encountered: BigramLanguageModel.__init__() got an unexpected keyword argument 'name'
This Pull Request introduces a new example demonstrating the implementation of a Generative Pretrained Transformer (GPT) using Keras. The example showcases how to build, train, and generate text with a GPT model, providing a comprehensive guide for users interested in natural language processing (NLP) and deep learning.