[llm_perf] add tp, kvcache and schedule support

bytedance / ByteMLPerf

AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and versatility of software and hardware.

https://bytemlperf.ai/

Apache License 2.0

188 stars 50 forks source link

[llm_perf] add tp, kvcache and schedule support #80

Closed suisiyuan closed 2 months ago

suisiyuan commented 2 months ago

add tp, kvcache and schedule support for chatglm2-6b model for gpu backend as demo project.