[Python] Add sentencepiece as installation requirement

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation

https://llm.mlc.ai/

Apache License 2.0

19.23k stars 1.58k forks source link

Closed MasterJH5574 closed 6 days ago

MasterJH5574 commented 6 days ago

This PR adds the sentencepiece package as a Python installation requirement for tokenizer coverage when running gen_config.