Closed ghost closed 1 year ago
BLAS is currently not supported for rwkv. They are working on a different parallel acceleration approach while you can follow in the rwkv.cpp repo
I was also just now wondering if that's possible. Thank you for that answer @LostRuins
Hi, I use an android device to run koboldcpp. Blas is working as expected, even with the new redpajama models.
However I am testing RWKV-4-Raven-3B-v11-Eng99-Other1-20230425-ctx4096-ggml-q5_1.bin and blas does not process the prompts, it's just regular processing.
I expect normal processing for low token prompts, but currently it's processing larger prompts (200 token) without blas.
Thank you for any direction on this issue.