search

Anush008 / fastembed-rs

Library for generating vector embeddings, reranking in Rust
https://docs.rs/fastembed
Apache License 2.0
264 stars 36 forks source link

feat: "bring your own" embeddings models #40

Closed jcorrie closed 6 months ago

jcorrie commented 6 months ago

On top of the existing facility to load embedding models from the huggingface hub, this exposes an interface for constituting a UserDefinedEmbeddingModel by passing in the bytes of the relevant files. (This is preferred over accepting pathbuf arguments, as it gives the user flexibility to load files from any source.)

Anush008 commented 6 months ago

Hi @jcorrie. Could you please document this in the README.md for more visibility

jcorrie commented 6 months ago

Hi @jcorrie. Could you please document this in the README.md for more visibility

Hi @Anush008 - yes, will do. I'm also looking at the suggestions made by Sourcery.

jcorrie commented 6 months ago

@Anush008 - I've added some blurb to the readme, but I didn't want to put too much detail in as to how a UserDefinedEmbeddingModel is constituted (it might be a bit noisy). I can put some more time into a demo if need be (I have some code from my working main.rs file I can repurpose).

Test also added for UserDefinedEmbeddingModel per sourcery suggestion - passes on my machine... I didn't take on board all of their suggestions but I've left some comments if you wanted to take a look. Again, happy to put some more time into resolving those if you think helpful (I just thought some looked a bit like overkill).

Anush008 commented 6 months ago

HI @jcorrie. We'll have to do a release to introduce the #44 fix. I'll include this PR too. We can work on the README.md later.

github-actions[bot] commented 6 months ago

:tada: This PR is included in version 3.3.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

jcorrie commented 6 months ago

HI @jcorrie. We'll have to do a release to introduce the #44 fix. I'll include this PR too. We can work on the README.md later.

It looks like this is all solved? Nothing for me to do other than the readme?