search

tomaarsen / attention_sinks

Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
https://huggingface.co/blog/tomaarsen/attention-sinks
Apache License 2.0
650 stars 41 forks source link

Error loading Qwen-1_8B #35

Open haiphong93 opened 8 months ago

haiphong93 commented 8 months ago

I used the default code to load Qwen-1_8B and got this error: RuntimeError: The size of tensor a (33) must match the size of tensor b (17) at non-singleton dimension 2