Could you please open-source the code for the calculation and visualization of the statistic information of KV Cache?

jy-yuan / KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

https://arxiv.org/abs/2402.02750

MIT License

121 stars 10 forks source link

Could you please open-source the code for the calculation and visualization of the statistic information of KV Cache? #2

Closed wln20 closed 2 months ago

wln20 commented 2 months ago

It's really pleasant to discover such a nice work! Could you please further provide the code to calculate and visualize the statistic information of KV Cache (like the following image), so it would be easy for future researchers to observe the pattern of KV Cache on more new models? Thanks a lot! 2024-03-05_130540

jy-yuan commented 2 months ago

We have added the visualization code along with instructions to vis.ipynb. Thank you for bringing this to our attention!

wln20 commented 2 months ago

Thanks for your attention! By the way, I got some trouble when using pip install -r requirements.txt to install the dependencies: . It seems to be a problem concerning == and =. Could you help fix it?

jy-yuan commented 2 months ago

Apologies for the error. We used conda to export the requirements and have since updated the README.