search

openppl-public / ppl.llm.serving

Apache License 2.0
122 stars 13 forks source link

[feature] sever support token in & out, and batch rsp #55

Closed Vincent-syr closed 3 months ago

Vincent-syr commented 3 months ago

[feature] sever support token in & out, and batch rsp