search

QwenLM / qwen.cpp

C++ implementation of Qwen-LM
Other
506 stars 40 forks source link

如何下载tiktoken_cpp #74

Open eswulei opened 5 months ago

eswulei commented 5 months ago

root@dsw-266011-58f575f6f4-tc9n4:/mnt/workspace/qwen.cpp/tests# RAYON_NUM_THREADS=1 python benchmark.py Traceback (most recent call last): File "/mnt/workspace/qwen.cpp/tests/benchmark.py", line 6, in import tiktoken_cpp ModuleNotFoundError: No module named 'tiktoken_cpp'

想测试一下benchmark，网上搜不到tiktoken_cpp这个包的下载方式