BBuf RWKV-World-HF-Tokenizer issues - Githubissues

BBuf / RWKV-World-HF-Tokenizer

31 stars 5 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

The tokenizer is 2.5x slower than other huggingface tokenizer and the original blinks world tokenizer

#6 cahya-wirawan opened 1 month ago
0
Is tokenization_rwkv5.py equivalent to tokenization_rwkv_world.py from huggingface?

#5 shiroko98 opened 1 month ago
2
how to create tokenizer.json

#4 xinyinan9527 opened 2 months ago
4
rwkv6 support flash linear attention

#3 BBuf closed 2 months ago
0
fixes eventually for HF repo and v6

#2 SmerkyG closed 3 months ago
22
Allowing the conversion of local file models

#1 PicoCreator closed 5 months ago
0