Crash when running with RWKV 5 (Raven)

guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.

https://llmfarm.site

MIT License

1.05k stars 62 forks source link

Crash when running with RWKV 5 (Raven) #41

Open sammcj opened 5 months ago

sammcj commented 5 months ago

When trying to load Raven (RWKV 5), the application crashes and exits.

There are no error messages displayed.

Is it possible simply that the RWKV package needs updating?

guinmoon commented 5 months ago

Hi I haven't updated rwkv for a long time. The version that is used in LLMFarm does not support rwkv 5. I didn't know that these models are still popular. I will try to update it in new releases.

sammcj commented 5 months ago

ah no worries at all, they have been making some interesting advancements with RWKV 4 and 5.

https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers

Not a big deal but might be nice to bump the libs regardless 😄