add v5 int8 by unifying float and int8 att

BlinkDL / ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

Apache License 2.0

9.39k stars 689 forks source link

Closed daquexian closed 11 months ago

daquexian commented 11 months ago

v5 1b5: fp16 3629MB, fp16i8 1979MB