Closed 3outeille closed 1 year ago
This is exactly what we need :) Please work on ChatRWKV
And please take a look at https://github.com/hahnyuan/RPTQ4LLM
And you only need quantization for matrix*vec (ignore all time_xxx stuff - they have to be in fp32. tiny amt of computation).
Referencing https://github.com/BlinkDL/ChatRWKV/pull/98 for any questions related to this topic
@BlinkDL Hi, I am willing to dedicate some time to implement GPTQ for RWKV, is that okay ?