search

abertsch72 / unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"
MIT License
1.05k stars 77 forks source link

Can't run the provided llama2 example #59

Open KerolosAtef opened 8 months ago

KerolosAtef commented 8 months ago

I got this error message while trying to run the prompt in the README file File "python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 402, in forward kv_seq_len += past_key_value.get_usable_length(kv_seq_len, self.layer_idx) AttributeError: 'list' object has no attribute 'get_usable_length'

KerolosAtef commented 8 months ago

@urialon

abertsch72 commented 8 months ago

Hi there-- what version of transformers are you running? This looks like an error inside the transformers library. Are you able to run llama without unlimiformer?

KerolosAtef commented 8 months ago

transformer version is 4.36.0 and also this happenes with the latest version 4.36.2. yes, it runs normally without unlimiformer

abertsch72 commented 8 months ago

downgrading to 4.27.0 should (likely) fix this for now; I'll look into the issue with the newer version

KerolosAtef commented 8 months ago

Thanks, it works with version 4.28.0

OswaldHe commented 3 months ago

Hi, is it possible to have a quick fix of this issue in the new version? Seems like downgrade transformers to a version before 4.35.0 has several rust compiler error. It's probably more convenient to use to most recent one. Thank you.