Regarding deployment using TF serving or Pytroch serving

Vamsi995 / Paraphrase-Generator

A paraphrase generator built using the T5 model which produces paraphrased English sentences.

MIT License

309 stars 66 forks source link

Regarding deployment using TF serving or Pytroch serving #14

Open chintanshrinath opened 3 years ago

chintanshrinath commented 3 years ago

Dear I want to deploy the model either on TF Serving or pytorch serving.

I have few dobuts, can you help for same. 1). How to use weights for tf_model.h5 to reduce the size of model. Or same for pytorch_model.bin 2) Or can you help me deploy in production mode.

Thanks

Vamsi995 commented 3 years ago

Please visit this link Vamsi/T5_Paraphrase_Paws. You can find the t5_model.h5 and the pytorch_model.bin files for this model. Maybe you can follow this guide for quantizing the weights https://spell.ml/blog/pytorch-quantization-X8e7wBAAACIAHPhT.