search

huggingface / setfit

Efficient few-shot learning with Sentence Transformers
https://hf.co/docs/setfit
Apache License 2.0
2.24k stars 221 forks source link

Serving SeTFiT models in torchserve #300

Open maccarini opened 1 year ago

maccarini commented 1 year ago

Im trying to deploy my setfit model in torchserver using a custom handler for this task. The thing is that im not being able to do this since im getting multiple errors while registering the model on the service. image I was wondering if this is actually possible since i tried to serve sentence-transformers models before successfully. Also to clarify, Im including setfit in requirements.txt

lsmith77 commented 1 year ago

would also be interested to learn about this.

puppetm4st3r commented 4 months ago

@maccarini can you resolve it? i'm looking a way to serv the model, I can temporarily add this models to my embeddings and re-ranking server repo but it is not a solution that makes the most of GPU resources usage, however it is a good starting point if we have no other option (for now)