search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.37k stars 90 forks source link

Can we have an example of pure C ++? #112

Open cgisky1980 opened 1 year ago

cgisky1980 commented 1 year ago

Can we have an example of pure C ++ ? python & pytorch is too big for the end users.

saharNooby commented 1 year ago

python & pytorch is too big for the end users.

I agree, but pure C++ implementation is blocked by implementing tokenizer, specifically -- Unicode normalization and Unicode regexes in 20B_tokenizer. Although RWKV World models are easier to support, since their tokenizer does not require Unicode libraries.

I myself have no plans of implementing the tokenizer in C, but I welcome PRs.

cgisky1980 commented 1 year ago

https://github.com/koute/rwkv_tokenizer rust

cgisky1980 commented 1 year ago

python & pytorch is too big for the end users.

I agree, but pure C++ implementation is blocked by implementing tokenizer, specifically -- Unicode normalization and Unicode regexes in 20B_tokenizer. Although RWKV World models are easier to support, since their tokenizer does not require Unicode libraries.

I myself have no plans of implementing the tokenizer in C, but I welcome PRs.

https://github.com/mlc-ai/tokenizers-cpp

saharNooby commented 1 year ago

@cgisky1980 Unfortunately, tokenizers-cpp will not work here:

You also need to turn on c++17 support