search

ELS-RD / transformer-deploy

Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
https://els-rd.github.io/transformer-deploy/
Apache License 2.0
1.64k stars 150 forks source link

How to deploy TrCOR #163

Closed SShimmyo closed 1 year ago

SShimmyo commented 1 year ago

Hi, I'm trying to deploy TrOCR (https://huggingface.co/microsoft/trocr-small-printed) based on encoder and decoder model by using triton inference server. However, I could not find any examples to deploy them. I would be great if you share example codes.

ayoub-louati commented 1 year ago

Hello @luenberger, we are working on T5 conversion (which is based on the encoder decoder architecture) in this PR: https://github.com/ELS-RD/transformer-deploy/pull/131 so you can take a look and you'll a find a very good example.

SShimmyo commented 1 year ago

Sorry for the late reply. Thank you for the information. I'l take a look at it.