How can I use opencompass benchmark tools to test ktransformers in long context?

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Apache License 2.0

726 stars 37 forks source link

How can I use opencompass benchmark tools to test ktransformers in long context? #91

Open AsVoider opened 1 month ago

AsVoider commented 1 month ago

The document tells that long context is supported only in local_chat now. If I can't use REST API, Is there any way to test ktransofrmers under opencompass framework locally?

qiyuxinlin commented 3 weeks ago

According to the official documentation of opencompass, we implemented a generate function, and then you need to refer to the source code in opencompass to modify the configuration of opencompass. The following is the official documentation: https://opencompass.org.cn/doc