The measurement results are not quite accurate as reported.

davidberenstein1957 / fast-sentence-transformers

Simply, faster, sentence-transformers

MIT License

140 stars 10 forks source link

The measurement results are not quite accurate as reported. #17

Closed BinhMinhs10 closed 5 months ago

BinhMinhs10 commented 6 months ago

The measurement results are not quite accurate as reported. For example, when I indexed 40,000 sentences using sentence-transformers on GPT, it took 97 seconds, but switching to fast-sentence-transformers with ONNX and GPU took 69 seconds. It's only about 1.4 times faster.

davidberenstein1957 commented 5 months ago

Sorry for the late response but I've got good news! Sentence Transformers has an awesome maintainer again @tomaarsen, along with awesome ONNX integrations through optimum, therefore I decided to archive this project. https://github.com/UKPLab/sentence-transformers. Take a look here https://www.philschmid.de/optimize-sentence-transformers.