search

mit-han-lab / streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks
https://arxiv.org/abs/2309.17453
MIT License
6.59k stars 361 forks source link

Evaluation code and dataset release inquiry #83

Open DerrickYLJ opened 3 months ago

DerrickYLJ commented 3 months ago

Hi StreamingLLM team,

In the paper, it includes generic perplexity test like below:

Screenshot 2024-06-18 at 8 09 09 PM

What code should I use if we want to perform the test? I wonder if and when the evaluation code and dataset for streaming are ready for release.

Thank you for your amazing work!