search

mybigday / llama.rn

React Native binding of llama.cpp
MIT License
300 stars 24 forks source link

RWKV support (sync llama.cpp) #77

Closed tomasmcm closed 14 hours ago

tomasmcm commented 1 month ago

Llama.cpp added support for RWKV models in https://github.com/ggerganov/llama.cpp/releases/tag/b3651 Can you sync the version llama.rn uses so that we can try those models? They are very lightweight and need a lot less RAM than transformers models, so it's a great fit for mobile devices.

a-ghorbani commented 1 week ago

Any chance this might get synced with the latest llama.cpp? I was going to give it a shot, but noticed a number of refactoring and changes in llama.cpp and didn’t want to risk messing up the codebase.

a-ghorbani commented 1 week ago

Kudos to @Vali-98 A synced version is available here: https://github.com/Vali-98/cui-llama.rn

Vali-98 commented 6 days ago

Hey there, just a reminder that cui-llama.rn currently lacks a lot of Swift / IOS integration for the new features added. I cannot guarantee that it'll work.