search

ELS-RD / transformer-deploy

Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
https://els-rd.github.io/transformer-deploy/
Apache License 2.0
1.64k stars 150 forks source link

Embedding with T5-Encoder #153

Open DA-L3 opened 1 year ago

DA-L3 commented 1 year ago

Hello,

thanks for this amazing framework.

I wanted to ask since I am interested in generating embeddings, is the usage of the T5EncoderModel integrated? As far as I can tell, only the special case with Sentence-Transformers is supported currently or am I missing something?

Thanks!

ayoub-louati commented 1 year ago

@CodyLDA Hello, We are currently working on integrating T5 model conversion so that it can be used for text generation. Once it is done, I think that generating embeddings should be much more easier following the T5 model example.