How can I use opencompass benchmark tools to test ktransformers in long context?

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

Apache License 2.0

741 stars 39 forks source link

How can I use opencompass benchmark tools to test ktransformers in long context? #91

Open AsVoider opened 2 months ago

AsVoider commented 2 months ago

The document tells that long context is supported only in local_chat now. If I can't use REST API, Is there any way to test ktransofrmers under opencompass framework locally?

qiyuxinlin commented 1 month ago

According to the official documentation of opencompass, we implemented a generate function, and then you need to refer to the source code in opencompass to modify the configuration of opencompass. The following is the official documentation: https://opencompass.org.cn/doc