search

Anush008 / fastembed-rs

Library for generating vector embeddings, reranking in Rust
https://docs.rs/fastembed
Apache License 2.0
285 stars 41 forks source link

Build Complexity, Musl Support? - Use of `ort` #117

Closed inadahime closed 2 months ago

inadahime commented 2 months ago

I'd like to start off with a thank you for developing this library - it's awesome, and so convenient! :)

Cutting to the chase: your implementation of fastembed uses ort to handle the underlying ONNX inference (i.e. the actual "embed" in "fastembed"). While this allows it to be performant on supported platforms, my understanding is that it leads to a more complex build process and makes targeting *-musl (e.g. x86_64-unknown-linux-musl, my go-to for portable Linux programs) platforms a chore: to do so, you must build the onnxruntime library for musl yourself, and then configure the ort build to use your local onnxruntime with ORT_LIB_LOCATION=/path/to/musl/onnxruntime/build during compilation time.

Are there any plans to switch the ONNX inference library to something other than ort, like sonos/tract, or would such a change be better suited to a fork?

Anush008 commented 2 months ago

Hey @inadahime. There are no plans as such.

inadahime commented 2 months ago

I see.