search

cpcdoy / rust-sbert

Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)
Apache License 2.0
106 stars 12 forks source link

Any plans to port utils like semantic_search? #17

Closed paulbricman closed 3 years ago

paulbricman commented 3 years ago

Hi @cpcdoy,

I was wondering whether there are any plans to also port some of the auxiliary code around the models from sentence-transformers, like semantic_search. This is probably one of the main use cases of both libraries, so many people might be interested in it in the future.

edit: I saw a contributor also works on https://github.com/lerouxrgd/ngt-rs which should do the trick. @lerouxrgd, an example of combining rust-sbert and ngt-rs would be really really useful for this!

lerouxrgd commented 3 years ago

Yes indeed rust-sbert and ngt-rs are meant to be used in tandem for semantic search. Basically all you have to do is to push the sbert embeddings (vectors of size 512) into an NGT index (you can check the doc for that but it's pretty straightforward).

paulbricman commented 3 years ago

Thanks @lerouxrgd for the quick reply! I'll look into it and comment a link to a combined implementation here soon.

paulbricman commented 3 years ago

Here is the really messy result: https://github.com/Psionica/Dual/blob/933e2f2e2a3ea9d401f32449c2d644d9e7766dd5/rust-backend/src/sbert_test.rs