search

strongio / keras-bert

A simple technique to integrate BERT from tf hub to keras
258 stars 108 forks source link

Anyone managed to get this to work with saved model? #4

Open veqtor opened 5 years ago

veqtor commented 5 years ago

I've trained a model that retunes a bertlayer but I can't seem to get it to export as a saved model properly... Any ideas?

AxeldeRomblay commented 5 years ago

Did you try to run the following code instead?

tf.keras.models.save_model( model, 'model', overwrite=True, include_optimizer=True )

JiahuiSophieHU commented 5 years ago

I can't load the model: model.save('/Documents/BERT/Sentiment_Classification_model_smallsiwetest.h5', overwrite=True, include_optimizer=True )

new_model = keras.models.load_model('Sentiment_Classification_model_smallsiwetest.h5') Traceback (most recent call last):

File "", line 1, in new_model = keras.models.load_model('Sentiment_Classification_model_smallsiwetest.h5')

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/saving.py", line 453, in load_wrapper return load_function(*args, **kwargs)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/saving.py", line 545, in load_model model = _deserialize_model(h5dict, custom_objects, compile)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/saving.py", line 243, in _deserialize_model model = model_from_config(model_config, custom_objects=custom_objects)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/saving.py", line 588, in model_from_config return deserialize(config, custom_objects=custom_objects)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/layers/init.py", line 168, in deserialize printable_module_name='layer')

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object list(custom_objects.items())))

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/network.py", line 1043, in from_config process_layer(layer_data)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/engine/network.py", line 1029, in process_layer custom_objects=custom_objects)

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/layers/init.py", line 168, in deserialize printable_module_name='layer')

File "/lib/miniconda/envs/conda-venv/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 140, in deserialize_keras_object ': ' + class_name)

ValueError: Unknown layer: BertLayer

smfullman commented 5 years ago

@JiahuiSophieHU have you tried instantiating the BertLayer class and then including it as a custom_object argument?

new_model = keras.models.load_model('Sentiment_Classification_model_smallsiwetest.h5',
                                     custom_objects={'BertLayer': BertLayer})
devinharia commented 5 years ago

@JiahuiSophieHU have you tried instantiating the BertLayer class and then including it as a custom_object argument?

new_model = keras.models.load_model('Sentiment_Classification_model_smallsiwetest.h5',
                                     custom_objects={'BertLayer': BertLayer})

I am facing a similar situation, not being able to load the saved model.

@smfullman tried this, but I am getting an error saying 'Shapes (768,) and (512, 768) are incompatible'.

PradyumnaGupta commented 4 years ago

@devinharia I am facing the same problem. Did you get the solution?

connormeaton commented 4 years ago

@veqtor I was not able to save the model until I wrote my own get_config.py function in the BertLayer class. Adding this code to the class made it run for me:

'''

def get_config(self):

    config = super(BertLayer, self).get_config().copy()
    config.update({
        'n_fine_tune_layers': self.n_fine_tune_layers,
        # 'trainable': self.trainable,
        # 'output_size': self.output_size,
        'pooling': self.pooling,
        'bert_path': self.bert_path,
    })

    return config

'''