search

microsoft / LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
https://llmlingua.com/
MIT License
4.42k stars 241 forks source link

[Question]: Is It Possible to Share the Code how to Do the evaluation #169

Open ZhichaoWang970201 opened 1 month ago

ZhichaoWang970201 commented 1 month ago

Describe the issue

I am wondering if you can share the evaluation code for "LongLLMLingua mitigates the 'lost in the middle' issue in LLMs, enhancing long-context information processing."

iofu728 commented 1 month ago

Hi @zhichaoWang970201, thanks for your support.

You can follow the instructions in longchat-13b-16k to run the NaturalQA benchmark.

ZhichaoWang970201 commented 1 month ago

Screenshot 2024-07-19 at 8 28 18 PM Thank you for your sharing.

In addition, the data used for generating answer is already very short. Is this what the author used for prompt compressing? If not, will the author add prompt to ask LLM to copy exact text from the input prompt? The evaluation file asks for exactly matching.

Thank you.

ZhichaoWang970201 commented 1 month ago

The problem is solved when reading more of the lost in the middle paper.