search

davidberenstein1957 / fast-sentence-transformers

Simply, faster, sentence-transformers
MIT License
140 stars 10 forks source link

Why is the original model required? #14

Closed mgoldenbe closed 5 months ago

mgoldenbe commented 1 year ago

I tried removing the original files from .cache and leaving only the .onnx file. However, when I ran the script it downloaded the original model again. How can I get rid of the dependency on the original model? In fact, given onnx, I should only need onnxruntime to make inferences. So, how do I get rid of the dependencies such as torch?

Sayed-Ameer commented 1 year ago

did you find a solution, for the whole model dependency?

antonmosin commented 8 months ago

This package not only converts the model to onnx format, but also instantiates it to run inference. To run inference, you need the tokenizer, which is expected in the cache_folder. The tokenizer is instantiated by the transformer library, which depends on torch, among other things.

davidberenstein1957 commented 5 months ago

Sorry for the late response but I've got good news! Sentence Transformers has an awesome maintainer again @tomaarsen, along with awesome ONNX integrations through optimum, therefore I decided to archive this project. https://github.com/UKPLab/sentence-transformers. Take a look here https://www.philschmid.de/optimize-sentence-transformers.