search

cpcdoy / rust-sbert

Rust port of sentence-transformers (https://github.com/UKPLab/sentence-transformers)
Apache License 2.0
106 stars 12 forks source link

Added device parameter to allow usage with MPS (Apple Silicon) and Vulkan #32

Closed Luxbit closed 2 months ago

cpcdoy commented 2 months ago

Hello @Luxbit , after review, this looks good to me. Thank you for your contribution!

cpcdoy commented 2 months ago

I'll publish the crate when rust-bert publishes its latest version so we can pin it in the Cargo.toml since publishing requires pinned versions. For now, you can use the latest tag v0.5.0 (link) from this repo when using this crate.

Luxbit commented 2 months ago

Hey @cpcdoy, thanks for maintaining :)

Yeah, the versioning is a bit messy. Sadly, tch v0.15 is needed to make MPS work. But it is worth it, it gives you a nice 3x performance increase on M1 Max