search

google-research / bert

TensorFlow code and pre-trained models for BERT
https://arxiv.org/abs/1810.04805
Apache License 2.0
38.25k stars 9.62k forks source link

Saving a model after pretraining on custom data. #1289

Open johnny-brav0 opened 2 years ago

johnny-brav0 commented 2 years ago

Currently, after running the run_pretraining.py the model checkpoints get saved. But is there any way I can save the entire model as a model.pb file so that in future it could be loaded directly as a KerasLayer with tf_hub?

Or is there a way of loading model.ckpt as a KerasLayer?

Please let me know if there is a way to do either. It'd be very helpful.

Thanks.

patyizsolt commented 2 years ago

Currently, after running the run_pretraining.py the model checkpoints get saved. But is there any way I can save the entire model as a model.pb file so that in future it could be loaded directly as a KerasLayer with tf_hub?

Or is there a way of loading model.ckpt as a KerasLayer?

Please let me know if there is a way to do either. It'd be very helpful.

Thanks.

gerwindekruijf commented 2 years ago

I don't know the way for converting it to a model.pb, but for the people that find this issue. You can use https://huggingface.co/docs/transformers/converting_tensorflow_models to convert tensorflow checkpoints into pytorch_model.bin

ritikranjan12 commented 2 years ago

@theProcrastinatr I think you can do like this

Save the entire model as a SavedModel.

!mkdir -p saved_model model.save('saved_model/my_model')

my_model directory

ls saved_model

Contains an assets folder, saved_model.pb, and variables folder.

ls saved_model/my_model

Rush97234 commented 1 year ago

Currently, after running the run_pretraining.py the model checkpoints get saved. But is there any way I can save the entire model as a model.pb file so that in future it could be loaded directly as a KerasLayer with tf_hub?

Or is there a way of loading model.ckpt as a KerasLayer?

Please let me know if there is a way to do either. It'd be very helpful.

Thanks.