search

abertsch72 / unlimiformer

Public repo for the NeurIPS 2023 paper "Unlimiformer: Long-Range Transformers with Unlimited Length Input"
MIT License
1.05k stars 77 forks source link

Unable to produce any output with llama 2 summarization example #28

Open cem2ran opened 1 year ago

cem2ran commented 1 year ago

I've tried running the llama 2 example in the readme in Colab but I've been unable to get any output.

Colab for reference: https://colab.research.google.com/drive/1XEnQmW7RbMh8BGPbFVoWSikKJ-1H_h8F?usp=sharing

Would appreciate any input on this. Thanks!

urialon commented 1 year ago

Hi @cem2ran ,

Thank you for your interest in our work.

I'm not sure. Did it just hang after loading the checkpoint? Did it print any stack trace when you stopped it?

Please note that faiss-gpu version 1.7.4 is required, and from your logs it seems that pip installs version 1.7.2.

Can you also try the 7B version of Llama? Maybe 13B is too much for Colab.

See also this issue https://github.com/abertsch72/unlimiformer/issues/25 where users managed to run this on Colab.

Let us know how it goes. Best, Uri