search

kyegomez / LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
https://discord.gg/qUtxnK2NMf
Apache License 2.0
663 stars 63 forks source link

The README usage code failed to run. #10

Closed LetianLee closed 11 months ago

LetianLee commented 11 months ago

I tried running the README usage code in my Colab environment with an A100 GPU, but it appears to have failed. Please see the attached screenshot and my Colab code: https://colab.research.google.com/drive/1wU-O7kKk_Frq9q-bhXE87e-wNE47YZqV?usp=sharing

image

Thanks!

LetianLee commented 11 months ago

I'm sorry, I noticed your comment stating that "pip install" does not work for the "flash-attn" library because it cannot be compiled due to custom CUDA Kernels, and they need to be built manually.