search

Anush008 / fastembed-rs

Library for generating vector embeddings, reranking in Rust
https://docs.rs/fastembed
Apache License 2.0
239 stars 32 forks source link

Optional openssl #70

Open joshniemela opened 2 months ago

joshniemela commented 2 months ago

OpenSSL can often be annoying to use as a build dependency and it should be possible to use the rustls package to replace openssl

Anush008 commented 2 months ago

We don't have it as a direct dependency though. Could you please elaborate?

joshniemela commented 2 months ago

If your system doesn't have OpenSSL installed, cargo build will throw an error. This most likely is happening due to hf-hub or some other dependency that downloads stuff, I was not able to find the cause whilst swiftly looking through the dependencies

Anush008 commented 2 months ago

This would need a fix in the upstream projects then.

joshniemela commented 2 months ago

Indeed, from my quick checks it seems it is either hf-hub or some other package which is using it for downloading data from the internet with reqwest or an analogue, these packages already support rustls so it should be possible to make some simple upstream changes.

mcmah309 commented 1 month ago

It is being pulled in by fastembed -> hf-hub -> native-tls -> openssl

mcmah309 commented 1 month ago

native-tls was removed here https://github.com/huggingface/hf-hub/commit/9d6502f5bc2e69061c132f523c76a76dad470477 Resultingly, the current hf-hub does not seem to have a dependency on openssl. So hopefully this issue will resolve when they release the next version.

It may be added back though https://github.com/huggingface/hf-hub/issues/56