search

kyegomez / LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
https://discord.gg/qUtxnK2NMf
Apache License 2.0
663 stars 63 forks source link

Any demo python I can play with? #5

Closed AK51 closed 11 months ago

AK51 commented 1 year ago

Hi,

I have installed LongNet in my Ubuntu with 4090. (Is it enough to run LongNet?) but when I type python example.py, there is error... I have tried "pip install torchscale", no help, same error.. >_< I am still reading the github and the paper, is there any demo python program I can play with first? Thx

Details (No error during LongNet installation)

pip install LongNet
Requirement already satisfied: LongNet in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (0.1.3)
Requirement already satisfied: torch in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (2.0.1)
Requirement already satisfied: einops in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.6.1)
Requirement already satisfied: flash-attn in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (1.0.5)
Requirement already satisfied: accelerate in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.20.3)
Requirement already satisfied: bitsandbytes in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.39.1)
Requirement already satisfied: fairscale in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.4.0)
Requirement already satisfied: timm in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.4.12)
Requirement already satisfied: flamingo-pytorch in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from LongNet) (0.1.2)
Requirement already satisfied: numpy>=1.17 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (1.25.0)
Requirement already satisfied: packaging>=20.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (23.1)
Requirement already satisfied: psutil in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (5.9.5)
Requirement already satisfied: pyyaml in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from accelerate->LongNet) (6.0)
Requirement already satisfied: filelock in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.12.2)
Requirement already satisfied: typing-extensions in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (4.7.1)
Requirement already satisfied: sympy in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (1.12)
Requirement already satisfied: networkx in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.1)
Requirement already satisfied: jinja2 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (3.1.2)
Requirement already satisfied: nvidia-cuda-nvrtc-cu11==11.7.99 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.99)
Requirement already satisfied: nvidia-cuda-runtime-cu11==11.7.99 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.99)
Requirement already satisfied: nvidia-cuda-cupti-cu11==11.7.101 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.101)
Requirement already satisfied: nvidia-cudnn-cu11==8.5.0.96 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (8.5.0.96)
Requirement already satisfied: nvidia-cublas-cu11==11.10.3.66 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.10.3.66)
Requirement already satisfied: nvidia-cufft-cu11==10.9.0.58 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (10.9.0.58)
Requirement already satisfied: nvidia-curand-cu11==10.2.10.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (10.2.10.91)
Requirement already satisfied: nvidia-cusolver-cu11==11.4.0.1 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.4.0.1)
Requirement already satisfied: nvidia-cusparse-cu11==11.7.4.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.4.91)
Requirement already satisfied: nvidia-nccl-cu11==2.14.3 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (2.14.3)
Requirement already satisfied: nvidia-nvtx-cu11==11.7.91 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (11.7.91)
Requirement already satisfied: triton==2.0.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torch->LongNet) (2.0.0)
Requirement already satisfied: setuptools in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->LongNet) (67.8.0)
Requirement already satisfied: wheel in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch->LongNet) (0.38.4)
Requirement already satisfied: cmake in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from triton==2.0.0->torch->LongNet) (3.26.4)
Requirement already satisfied: lit in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from triton==2.0.0->torch->LongNet) (16.0.6)
Requirement already satisfied: einops-exts in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from flamingo-pytorch->LongNet) (0.0.4)
Requirement already satisfied: ninja in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from flash-attn->LongNet) (1.11.1)
Requirement already satisfied: torchvision in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from timm->LongNet) (0.15.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from jinja2->torch->LongNet) (2.1.3)
Requirement already satisfied: mpmath>=0.19 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from sympy->torch->LongNet) (1.3.0)
Requirement already satisfied: requests in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torchvision->timm->LongNet) (2.31.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from torchvision->timm->LongNet) (10.0.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (2.0.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/ak/anaconda3/envs/LongNet/lib/python3.9/site-packages (from requests->torchvision->timm->LongNet) (2023.5.7)

python example.py 
Traceback (most recent call last):
  File "/media/ak/HD/LongNet/example.py", line 3, in <module>
    from LongNet import DilatedAttention
  File "/media/ak/HD/LongNet/LongNet/__init__.py", line 6, in <module>
    from LongNet.attention import DilatedAttention
  File "/media/ak/HD/LongNet/LongNet/attention.py", line 6, in <module>
    from LongNet.torchscale import XPOS, RelativePositionBias
ImportError: cannot import name 'XPOS' from 'LongNet.torchscale' (unknown location)
kyegomez commented 12 months ago

@AK51 Hey pleaase excuse us, can you try again when you can please?

kuehno commented 12 months ago

@AK51 I was having the same issue initially. When installing LongNet using pip, it will also install torchscale. In the LongNet repository however, torchscale is forked and modified slightly in the init.py.

See original init.py from torchscale: https://github.com/microsoft/torchscale/blob/main/torchscale/__init__.py

In order to fix it you would need to clone the LongNet repository (which contains the respective changes) or clone + install torchscale to your working directory and modify the init.py yourself.

AK51 commented 12 months ago

Mm... I tried to clone the LongNet repo again but I can not find cd flash_attn

May I have a straight forward installation instruction. Thx I can wait.

kyegomez commented 12 months ago

@kuehno Yeah torchscale was a big issue, we've fixed that now.

And, @AK51 and yeah we removed it as we made our own custom implementation for flash attention here: https://github.com/kyegomez/LongNet/blob/master/LongNet/attend.py