search

openppl-public / ppl.llm.serving

Apache License 2.0
122 stars 13 forks source link

[feature][refactor] support more tokenizer, optimize stream chat proc… #22

Closed Vincent-syr closed 10 months ago

Vincent-syr commented 10 months ago

[feature][refactor] support more tokenizer, optimize stream chat procudure, use static thread pool to substitute raw thread pool