search

FasterDecoding / SnapKV

141 stars 4 forks source link

maybe a bug in `update_kv` function #3

Open HarryWu99 opened 2 months ago

HarryWu99 commented 2 months ago

https://github.com/FasterDecoding/SnapKV/blob/ea655b18061313e088879bd2b4a3e3c0c2dc2e21/snapkv_utils.py#L50

In update_kv function, instead of using the function's arguments attention_mask, this variable is overridden.

leeyeehoo commented 2 months ago

Just refactored the codebase. I think the arg is dummy and will remove it in the future update. You can test to see if there is any bug! Thx!