search

mlc-ai / tokenizers-cpp

Universal cross-platform tokenizers binding to HF and sentencepiece
Apache License 2.0
211 stars 47 forks source link

fix rwkv world tokenzier #19

Closed BBuf closed 7 months ago

BBuf commented 7 months ago

修复后rwkv5 q4f16_2效果：

图片

修复前：

图片