How to reproduce GEAR on Mistral models

opengear-project / GEAR

GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

MIT License

128 stars 10 forks source link

How to reproduce GEAR on Mistral models #12

Closed CUHKSZzxy closed 1 month ago

CUHKSZzxy commented 1 month ago

Thanks for your excellent work!

May I ask for the scripts and examples to reproduce GEAR results on MMLU, using Mistral models? It is difficult for me to find related scripts and the usage of different configuration / parameters.

HaoKang-Timmy commented 1 month ago

We have updated sh files to run llama models on Aqua,GSM8k and BBH datasets

CUHKSZzxy commented 1 month ago

Thanks for your reply, but my concern is how to reproduce results on Mistral models, thanks!