search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.37k stars 90 forks source link

Support for MPI #116

Open cgisky1980 opened 1 year ago

cgisky1980 commented 1 year ago

https://github.com/ggerganov/ggml/pull/340

https://github.com/ggerganov/llama.cpp/pull/2099

The project hasn't been updated for a month, so I'll find some work for someone to do

theycallmeloki commented 1 year ago

@cgisky1980 I would like to pick this up continuing parallelly from ggerganov/llama.cpp#2164

I believe rwkv might be more optimized for the use case I am on currently so if you can boil it a bit to generic steps I would need to take to complete it, I'll try to pick it up