search

mdrokz / rust-llama.cpp

LLama.cpp rust bindings
https://crates.io/crates/llama_cpp_rs/
MIT License
290 stars 42 forks source link

Maintance and improvements #37

Open philschmid opened 4 months ago

philschmid commented 4 months ago

@mdrokz are you planning to main this project? I saw it uses a pretty old llama.cpp version.

mdrokz commented 4 months ago

@mdrokz are you planning to main this project? I saw it uses a pretty old llama.cpp version.

I'm not getting time these days due to my job if somebody wants to maintain this project im happy to add them in the repo. I also merge any PR's when i get time.

philschmid commented 4 months ago

Thank you for responding! I saw edgen starting their on fork at https://github.com/edgenai/llama_cpp-rs/tree/main, which is inspired by your work https://github.com/edgenai/llama_cpp-rs/blob/847d4f5d104c23b87a925618027942313badec2c/crates/llama_cpp_sys/build.rs#L8

Maybe we could unify all the rust efforts into a single package. pinging @pedro-devv and @scriptis

pedro-devv commented 4 months ago

We will keep developing our own bindings for the foreseeable future. The bindings from this repository aren't exactly what we are looking for in our projects, which is why we chose to make our own to begin with. Regardless, I'll take this chance to thank you for your work done on these bindings! They were a great starting point for us, especially the compilation side of things.

philschmid commented 4 months ago

@pedro-devv thank you for the respond! So it might make more sense to redirect people than to your bindings if you plan to maintain them and @mdrokz is not having the time at the moment.

For the Rust community it would be very nice if we can find 1 repo where all can contribute and work on.

mdrokz commented 4 months ago

Thank you for responding! I saw edgen starting their on fork at https://github.com/edgenai/llama_cpp-rs/tree/main, which is inspired by your work https://github.com/edgenai/llama_cpp-rs/blob/847d4f5d104c23b87a925618027942313badec2c/crates/llama_cpp_sys/build.rs#L8

Maybe we could unify all the rust efforts into a single package. pinging @pedro-devv and @scriptis

Yeah edgenai's project looks more advanced and cleaner than mine it might be an better alternative to my bindings

mdrokz commented 4 months ago

We will keep developing our own bindings for the foreseeable future. The bindings from this repository aren't exactly what we are looking for in our projects, which is why we chose to make our own to begin with. Regardless, I'll take this chance to thank you for your work done on these bindings! They were a great starting point for us, especially the compilation side of things.

Thank you! Can i ask what was missing in my bindings ? and what you were looking for ?