jean-pierreBoth / hnswlib-rs

Rust implementation of the HNSW algorithm (Malkov-Yashunin)
Other
157 stars 22 forks source link

improvement: make HnswIo Sync so it can be shared between threads #14

Closed MathGeniusJodie closed 9 months ago

MathGeniusJodie commented 9 months ago

right now I get this error

std::cell::RefCell<usize>` cannot be shared between threads safely
within `(std::string::String, hnsw_rs::hnswio::HnswIo)`, the trait `std::marker::Sync` is not implemented for `std::cell::RefCell<usize>`
if you want to do aliasing and mutation between multiple threads, use `std::sync::RwLock` instead
required because it appears within the type `(String, HnswIo)
jean-pierreBoth commented 9 months ago

I can put nb_point_loaded an AtomicUsize but reading data file will still be serial. Will that really change the use of hnswio?

Le sam. 27 janv. 2024 à 03:39, MathGeniusJodie @.***> a écrit :

right now I get this error std::cell::RefCell cannot be shared between threads safely within (std::string::String, hnsw_rs::hnswio::HnswIo), the trait std::marker::Sync is not implemented for std::cell::RefCell if you want to do aliasing and mutation between multiple threads, use std::sync::RwLock instead required because it appears within the type (String, HnswIo)

— Reply to this email directly, view it on GitHub https://github.com/jean-pierreBoth/hnswlib-rs/issues/14, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEXM7NI7XIL6O5HACRTUDODYQRSHJAVCNFSM6AAAAABCNAGVXGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYDGMJUG4YTMNY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

MathGeniusJodie commented 9 months ago

it would be incredibly valuable for async and for integrating with existing multithreaded systems, in fact I'm considering forking this library just for this feature

jean-pierreBoth commented 9 months ago

pushed on https://github.com/jean-pierreBoth/hnswlib-rs (not yet on crates.io) Thanks