search

OpenPPL / ppl.llm.serving

Apache License 2.0
123 stars 13 forks source link

[feature][refactor] support more tokenizer, optimize stream chat proc… #22

Closed Vincent-syr closed 1 year ago

Vincent-syr commented 1 year ago

[feature][refactor] support more tokenizer, optimize stream chat procudure, use static thread pool to substitute raw thread pool