search

Dao-AILab / flash-attention

Fast and memory-efficient exact attention
BSD 3-Clause "New" or "Revised" License
14.22k stars 1.33k forks source link

flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol #853

Open yangguoming opened 8 months ago

yangguoming commented 8 months ago

20240226155758

yangguoming commented 8 months ago

how do I fix with it?

Avelina9X commented 8 months ago

Which version of pytorch and CUDA are you using? Which version of flash attention are you using?

yangguoming commented 8 months ago

torch2.1.0 cuda12.1 and flash-attn 2.4.2

nspyf commented 8 months ago

I encountered a similar error while attempting to install the package using the command pip install flash_attn-2.5.5+cu118torch2.2cxx11abiTRUE-cp38-cp38-linux_x86_64.whl However, the issue was resolved by using the command pip install flash_attn-2.5.5+cu118torch2.2cxx11abiFALSE-cp38-cp38-linux_x86_64.whl PyTorch version 2.2.0 CUDA version 11.8 Python version 3.8.18

ghost commented 5 months ago

I also encountered the issue. Could you please advise how to fix the issue? Thanks.

import torch
import flash_attn_2_cuda as flash_attn_cuda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /XXXXX/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

My environment is as follows:

ghost commented 5 months ago

I just found a solution from here: https://github.com/Dao-AILab/flash-attention/issues/620 CUDA needs to be downgraded to version 11, then install this module instead: https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.8/flash_attn-2.5.8+cu118torch2.3cxx11abiFALSE-cp311-cp311-linux_x86_64.whl

pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.8/flash_attn-2.5.8+cu118torch2.3cxx11abiFALSE-cp311-cp311-linux_x86_64.whl
timothelaborie commented 5 months ago

for anyone who is here after trying to setup unsloth on runpod or primeintellect, both of those services have an "axolotl" docker image where flash-attn is already working. Use that, then run:

pip install "unsloth[cu118-ampere-torch211] @ git+https://github.com/unslothai/unsloth.git" --no-dependencies --disable-pip-version-check

pip install trl --no-dependencies --disable-pip-version-check --upgrade

pip install transformers --no-dependencies --disable-pip-version-check --upgrade