Closed SysuJayce closed 2 years ago
cc @Rocketknight1
The models from transformers
should be saved using the save_pretrained
method. For TensorFlow models, you can obtain the result as a SavedModel
by using the saved_model
keyword argument:
model.save_pretrained("/tmp/model", saved_model=True)
@LysandreJik Thx for your reply. I still have some questions, please take a look.
What is the diff between model.save_pretrained(path, saved_model=True)
and model.save(path)
? If I can save mode with the latter, should I change to the former?
Besides, I notice that when saving model, warning occurs. Is it a warning that can be ignored safely? What can I do to fix the warning?
WARNING:absl:Found untraced functions such as embeddings_layer_call_and_return_conditional_losses, embeddings_layer_call_fn, encoder_layer_call_and_return_conditional_losses, encoder_layer_call_fn, pooler_layer_call_and_return_conditional_losses while saving (showing 5 of 1055). These functions will not be directly callable after loading.
WARNING:absl:Found untraced functions such as embeddings_layer_call_and_return_conditional_losses, embeddings_layer_call_fn, encoder_layer_call_and_return_conditional_losses, encoder_layer_call_fn, pooler_layer_call_and_return_conditional_losses while saving (showing 5 of 1055). These functions will not be directly callable after loading.
Hi @SysuJayce yes, you can ignore that warning. That warning can also pop up when saving a large model using model.save
, it's just telling you that the model has some methods that weren't saved/traced, which is normal. Don't worry about fixing it.
Also, in general we don't support model.save
because our 'standard' way of saving/loading models is to use save_pretrained
and then from_pretrained
to load it again, like model = TFBertForMultilabelClassification.from_pretrained("/tmp/model")
.
The reasons we do this instead of using SavedModel
are bit long and confusing - the key issue is that SavedModel
saves the model graph but not necessarily all the code, and so you won't necessarily have all the capabilities of the model when you reload the SavedModel
file - this is basically what the warning is telling you. If you just want to call the model with new data, then SavedModel
should work fine for you, but try passing the file path to the from_pretrained
method if you want to load it perfectly and have it work just like it did originally.
Hi @SysuJayce yes, you can ignore that warning. That warning can also pop up when saving a large model using
model.save
, it's just telling you that the model has some methods that weren't saved/traced, which is normal. Don't worry about fixing it.Also, in general we don't support
model.save
because our 'standard' way of saving/loading models is to usesave_pretrained
and thenfrom_pretrained
to load it again, likemodel = TFBertForMultilabelClassification.from_pretrained("/tmp/model")
.The reasons we do this instead of using
SavedModel
are bit long and confusing - the key issue is thatSavedModel
saves the model graph but not necessarily all the code, and so you won't necessarily have all the capabilities of the model when you reload theSavedModel
file - this is basically what the warning is telling you. If you just want to call the model with new data, thenSavedModel
should work fine for you, but try passing the file path to thefrom_pretrained
method if you want to load it perfectly and have it work just like it did originally.
Hello @Rocketknight1 ,
In my situation, I'd like to train with huggingface transformers, and serve with tensorflow serving.
Therefore, I want to save and load the trained model with model.save()
.
Now I know that the preferred way to save and load trained model is from_pretrained()
and save_pretrained()
, but we can try tensorflow's original model.save()
and model.load()
.
Maybe I can save trained model with model.save_pretrained(path, saved_model=True)
and serve with tensorflow serving? What's your advice?
Thanks for your reply.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
Environment info
transformers
version: 4.10.3Who can help
@LysandreJik @Rocketknight1
Information
Model I am using (Bert, XLNet ...): roberta
The problem arises when using:
The tasks I am working on is:
To reproduce
Steps to reproduce the behavior:
Error messages:
Expected behavior
model can save in saved_model format without error