search

FasterDecoding / SnapKV

141 stars 4 forks source link

expanded_attn_mask = causal_4d_mask.masked_fill(expanded_attn_mask.bool(), torch.finfo(dtype).min) RuntimeError: The size of tensor a (3509) must match the size of tensor b (7017) at non-singleton dimension 3 #13

Closed seeyourcell closed 1 month ago

seeyourcell commented 1 month ago

I use transformers==4.27.0 ，why has this error？

RuntimeError: The size of tensor a (3509) must match the size of tensor b (7017) at non-singleton dimension 3

seeyourcell commented 1 month ago

expanded_attn_mask = causal_4d_mask.masked_fill(expanded_attn_mask.bool(), torch.finfo(dtype).min) RuntimeError: The size of tensor a (3509) must match the size of tensor b (7017) at non-singleton dimension 3

pengshuang commented 3 weeks ago

Hello, I met the same problem, how do you solve it?

Zhutianyi7230 commented 3 weeks ago

I meet the same question when I set use_flasg_attention_2=False