search

L1aoXingyu / llm-infer-bench

11 stars 0 forks source link

[feature request] Could you please nicely add the benchmark results of lmdelopy. #1

Open huliang2016 opened 1 year ago

huliang2016 commented 1 year ago

LMDeploy is a toolkit for compressing, deploying, and serving LLMs. It also supports llama and llama-2.

Wondering if it could be the fastest servering tools.

L1aoXingyu commented 1 year ago

Maybe I will test it if I have additional time.