How to speed up inference generation for sentence vectors？

UKPLab / sentence-transformers

Multilingual Sentence & Image Embeddings with BERT

https://www.SBERT.net

Apache License 2.0

14.72k stars 2.43k forks source link

Open yangtianyu92 opened 4 years ago

yangtianyu92 commented 4 years ago

Is it possible to use the FP16 approach to speed up the inference of the model? How can I do it if I want to accelerate so much? Thank you.

yangtianyu92 commented 4 years ago

self.half() , add this code in sentence-transformers.py def init last line