search

Dao-AILab / flash-attention

Fast and memory-efficient exact attention
BSD 3-Clause "New" or "Revised" License
11.93k stars 1.06k forks source link

ImportError: undefined symbol #854

Open austinmw opened 4 months ago

austinmw commented 4 months ago

steps taken

%pip uninstall torch transformers datasets bitsandbytes trl perft flash-attn -y
%pip install -Uq transformers[torch] datasets
%pip install -Uq bitsandbytes trl peft
%pip install flash-attn --no-build-isolation

from flash_attn import flash_attn_qkvpacked_func

error

ImportError: /opt/conda/envs/pytorch/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE

pip freeze

about-time==4.2.1
accelerate==0.26.1
aiohttp==3.9.1
aiosignal==1.3.1
alive-progress==3.1.5
aniso8601==9.0.1
annotated-types @ file:///home/conda/feedstock_root/build_artifacts/annotated-types_1696634205638/work
ansi2html==1.9.1
anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1702909220329/work
argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1692818318753/work
argon2-cffi-bindings @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi-bindings_1695386546427/work
arrow @ file:///home/conda/feedstock_root/build_artifacts/arrow_1696128962909/work
asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1698341106958/work
async-lru @ file:///home/conda/feedstock_root/build_artifacts/async-lru_1690563019058/work
async-timeout==4.0.3
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
autograd==1.6.2
awscli @ file:///home/conda/feedstock_root/build_artifacts/awscli_1705260438297/work
Babel @ file:///home/conda/feedstock_root/build_artifacts/babel_1702422572539/work
backoff==2.2.1
beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1680888073205/work
bitsandbytes==0.42.0
bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1696630167146/work
blinker==1.7.0
blis==0.7.11
bokeh @ file:///home/conda/feedstock_root/build_artifacts/bokeh_1704748008822/work
boto3==1.34.34
botocore==1.34.34
Brotli @ file:///home/conda/feedstock_root/build_artifacts/brotli-split_1695989787169/work
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
captum @ file:///home/conda/feedstock_root/build_artifacts/captum_1676431791267/work
catalogue==2.0.10
certifi @ file:///home/conda/feedstock_root/build_artifacts/certifi_1700303426725/work/certifi
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1696001684923/work
chardet==5.2.0
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1698833585322/work
click==8.1.7
cloudpathlib==0.16.0
cloudpickle==2.2.1
cma==3.2.2
cmake==3.25.0
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1602866480661/work
comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1704278392174/work
confection==0.1.4
contextlib2==21.6.0
contourpy @ file:///home/conda/feedstock_root/build_artifacts/contourpy_1699041363598/work
cryptography==42.0.1
cycler @ file:///home/conda/feedstock_root/build_artifacts/cycler_1696677705766/work
cymem==2.0.8
dataclasses-json==0.6.3
datasets==2.17.1
debugpy @ file:///home/conda/feedstock_root/build_artifacts/debugpy_1695534290310/work
decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work
Deprecated==1.2.14
dill==0.3.7
dirtyjson==1.0.8
distro==1.9.0
docker==7.0.0
docstring-parser==0.15
docutils @ file:///home/conda/feedstock_root/build_artifacts/docutils_1667993608396/work
dparse==0.6.3
einops==0.7.0
emoji==2.10.1
entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1704921103267/work
executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1698579936712/work
faiss==1.7.4
fastapi==0.95.2
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1703780968325/work/dist
filelock @ file:///home/conda/feedstock_root/build_artifacts/filelock_1698714947081/work
filetype==1.2.0
FlagEmbedding==1.1.9
flash-attn==2.5.5
Flask==3.0.0
Flask-RESTful==0.3.10
fonttools @ file:///home/conda/feedstock_root/build_artifacts/fonttools_1704979835861/work
fqdn @ file:///home/conda/feedstock_root/build_artifacts/fqdn_1638810296540/work/dist
frozenlist==1.4.1
fsspec==2023.10.0
future==0.18.3
fuzzywuzzy==0.18.0
gmpy2 @ file:///home/conda/feedstock_root/build_artifacts/gmpy2_1666808654411/work
google-pasta==0.2.0
grapheme==0.6.0
greenlet==3.0.3
gym==0.26.2
gym-notices==0.0.8
h11==0.14.0
html2text==2020.1.16
httpcore==1.0.2
httpx==0.26.0
huggingface-hub==0.21.1
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1701026962277/work
imageio @ file:///home/conda/feedstock_root/build_artifacts/imageio_1702571712725/work
importlib-metadata==6.11.0
importlib-resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1699364556997/work
ipykernel==6.29.0
ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1704718870316/work
ipython-genutils==0.2.0
ipywidgets @ file:///home/conda/feedstock_root/build_artifacts/ipywidgets_1694607144474/work
isoduration @ file:///home/conda/feedstock_root/build_artifacts/isoduration_1638811571363/work/dist
itsdangerous==2.1.2
jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1696326070614/work
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1704966972576/work
jmespath @ file:///home/conda/feedstock_root/build_artifacts/jmespath_1655568249366/work
joblib==1.3.2
json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1688248289187/work
jsonpatch==1.33
jsonpath-python==1.0.6
jsonpickle==3.0.2
jsonpointer @ file:///home/conda/feedstock_root/build_artifacts/jsonpointer_1695397238043/work
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema-meta_1700159890288/work
jsonschema-specifications @ file:///tmp/tmpkv1z7p57/src
jupyter-events @ file:///home/conda/feedstock_root/build_artifacts/jupyter_events_1699285872613/work
jupyter-lsp @ file:///home/conda/feedstock_root/build_artifacts/jupyter-lsp-meta_1701091994466/work/jupyter-lsp
jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1699283905679/work
jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1704727030956/work
jupyter_server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1704986851950/work
jupyter_server_terminals @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_terminals_1703611053195/work
jupyterlab @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_1703881392925/work
jupyterlab-widgets @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_widgets_1694598704522/work
jupyterlab_pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1700744013163/work
jupyterlab_server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server-split_1700310846957/work
kiwisolver @ file:///home/conda/feedstock_root/build_artifacts/kiwisolver_1695379902431/work
langchain==0.1.5
langchain-community==0.0.17
langchain-core==0.1.16
langcodes==3.3.0
langdetect==1.0.9
langsmith==0.0.83
Levenshtein==0.24.0
lit==15.0.7
llama-hub==0.0.77
llama-index==0.9.45.post1
llvmlite==0.41.1
LM_Cocktail==0.0.4
lxml==5.1.0
markdown-it-py==3.0.0
MarkupSafe @ file:///home/conda/feedstock_root/build_artifacts/markupsafe_1695367434228/work
marshmallow==3.20.2
matplotlib @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-suite_1700509477637/work
matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
mdurl==0.1.2
mistune @ file:///home/conda/feedstock_root/build_artifacts/mistune_1698947099619/work
mpmath @ file:///home/conda/feedstock_root/build_artifacts/mpmath_1678228039184/work
msgpack==1.0.7
multidict==6.0.4
multiprocess==0.70.15
munkres==1.1.4
murmurhash==1.0.10
mypy-extensions==1.0.0
nbclassic @ file:///home/conda/feedstock_root/build_artifacts/nbclassic_1683202081046/work
nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1684790896106/work
nbconvert @ file:///home/conda/feedstock_root/build_artifacts/nbconvert-meta_1704986998851/work
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1690814868471/work
nest_asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1705352640985/work
networkx @ file:///home/conda/feedstock_root/build_artifacts/networkx_1698504735452/work
ninja==1.11.1.1
nltk==3.8.1
notebook @ file:///home/conda/feedstock_root/build_artifacts/notebook_1680870634737/work
notebook_shim @ file:///home/conda/feedstock_root/build_artifacts/notebook-shim_1682360583588/work
numba==0.58.1
numpy @ file:///home/conda/feedstock_root/build_artifacts/numpy_1704280375097/work/dist/numpy-1.26.3-cp310-cp310-linux_x86_64.whl#sha256=d6b1cccf494a7701b8dd18911cff7c805fd90e8f4a24cde7c8e40f7805c0765d
nvgpu==0.10.0
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-nccl-cu12==2.19.3
nvidia-nvjitlink-cu12==12.3.101
nvidia-nvtx-cu12==12.1.105
openai==1.10.0
overrides @ file:///home/conda/feedstock_root/build_artifacts/overrides_1691338815398/work
packaging==23.2
pandas @ file:///home/conda/feedstock_root/build_artifacts/pandas_1702057131119/work
pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work
parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
pathos==0.3.1
patsy @ file:///home/conda/feedstock_root/build_artifacts/patsy_1704469236901/work
peft==0.8.2
pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1667297516076/work
pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
pillow @ file:///home/conda/feedstock_root/build_artifacts/pillow_1704252020178/work
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1701708255999/work
plotly==5.18.0
pox==0.3.3
ppft==1.7.6.7
preshed==3.0.9
prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1700579315247/work
prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1702399386289/work
protobuf==4.25.2
psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1702833087123/work
ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
pyaml==23.12.0
pyarrow==14.0.2
pyarrow-hotfix==0.6
pyasn1 @ file:///home/conda/feedstock_root/build_artifacts/pyasn1_1701287008248/work
pybind11 @ file:///home/conda/feedstock_root/build_artifacts/pybind11-split_1695367014606/work
pybind11-global @ file:///home/conda/feedstock_root/build_artifacts/pybind11-split_1695367014606/work
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
pydantic==1.10.13
pydantic_core @ file:///home/conda/feedstock_root/build_artifacts/pydantic-core_1703318464269/work
pyfunctional==1.4.3
pygame==2.5.2
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1700607939962/work
pymoo==0.6.1.1
pynvml==11.5.0
pyparsing @ file:///home/conda/feedstock_root/build_artifacts/pyparsing_1690737849915/work
pypdf==4.0.1
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
python-iso639==2024.1.2
python-json-logger @ file:///home/conda/feedstock_root/build_artifacts/python-json-logger_1677079630776/work
python-Levenshtein==0.24.0
python-magic==0.4.27
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1693930252784/work
pyvis==0.3.2
PyYAML @ file:///home/conda/feedstock_root/build_artifacts/pyyaml_1695373428874/work
pyzmq @ file:///home/conda/feedstock_root/build_artifacts/pyzmq_1701783169502/work
rapidfuzz==3.6.1
ray==2.9.1
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1704489226496/work
regex==2023.12.25
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1684774241324/work
retrying==1.3.4
rfc3339-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3339-validator_1638811747357/work
rfc3986-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3986-validator_1598024191506/work
rich==13.7.0
rpds-py @ file:///home/conda/feedstock_root/build_artifacts/rpds-py_1705159800683/work
rsa @ file:///home/conda/feedstock_root/build_artifacts/rsa_1614171254180/work
ruamel.yaml==0.18.5
ruamel.yaml.clib==0.2.8
s3fs==0.4.2
s3transfer @ file:///home/conda/feedstock_root/build_artifacts/s3transfer_1703197439685/work
safetensors==0.4.2
sagemaker==2.206.0
schema==0.7.5
scikit-learn==1.3.2
SciPy @ file:///home/conda/feedstock_root/build_artifacts/scipy-split_1700812469549/work/dist/scipy-1.11.4-cp310-cp310-linux_x86_64.whl#sha256=136e231ccb8768e60c17ed60f2c2423262d3dfd8136f373e715db9dd77617e41
seaborn @ file:///home/conda/feedstock_root/build_artifacts/seaborn-split_1704312987503/work
Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1682601222253/work
sentence-transformers==2.2.2
sentencepiece==0.1.99
shap==0.44.0
shtab==1.7.0
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
slicer==0.0.7
smart-open==6.4.0
smclarify==0.5
smdebug-rulesconfig==1.0.1
sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1662051266223/work
sortedcontainers==2.4.0
soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1693929250441/work
spacy==3.7.2
spacy-legacy==3.0.12
spacy-loggers==1.0.5
SQLAlchemy==2.0.25
srsly==2.4.8
stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work
starlette==0.27.0
statsmodels @ file:///home/conda/feedstock_root/build_artifacts/statsmodels_1702575356319/work
sympy @ file:///home/conda/feedstock_root/build_artifacts/sympy_1684180540116/work
syne-tune==0.10.0
tabulate==0.9.0
tblib==2.0.0
tenacity==8.2.3
termcolor==2.4.0
terminado @ file:///home/conda/feedstock_root/build_artifacts/terminado_1699810101464/work
thefuzz==0.22.1
thinc==8.2.2
threadpoolctl==3.2.0
tiktoken==0.5.2
tinycss2 @ file:///home/conda/feedstock_root/build_artifacts/tinycss2_1666100256010/work
tokenizers==0.15.2
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1644342247877/work
torch==2.2.1
torch-model-archiver @ file:///usr/share/miniconda/conda-bld/torch-model-archiver_1675893865207/work
torch-workflow-archiver @ file:///usr/share/miniconda/conda-bld/torch-workflow-archiver_1697147840057/work
torchdata @ file:///opt/conda/conda-bld/torchdata_1696637783505/work
torchserve @ file:///usr/share/miniconda/conda-bld/torchserve_1697147627059/work
torchtext==0.16.0
tornado @ file:///home/conda/feedstock_root/build_artifacts/tornado_1695373560918/work
tqdm @ file:///home/conda/feedstock_root/build_artifacts/tqdm_1691671248568/work
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1675110562325/work
transformers==4.38.1
triton==2.2.0
trl==0.7.11
typer==0.9.0
types-python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/types-python-dateutil_1704512562698/work
typing-inspect==0.9.0
typing-utils @ file:///home/conda/feedstock_root/build_artifacts/typing_utils_1622899189314/work
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1702176139754/work
tyro==0.7.3
tzdata @ file:///home/conda/feedstock_root/build_artifacts/python-tzdata_1703878702368/work
ujson==5.9.0
unicodedata2 @ file:///home/conda/feedstock_root/build_artifacts/unicodedata2_1695847980273/work
unstructured==0.12.3
unstructured-client==0.16.0
uri-template @ file:///home/conda/feedstock_root/build_artifacts/uri-template_1688655812972/work/dist
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1697813446430/work
uvicorn==0.22.0
wasabi==1.1.2
wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1704731205417/work
weasel==0.3.4
webcolors @ file:///home/conda/feedstock_root/build_artifacts/webcolors_1679900785843/work
webencodings @ file:///home/conda/feedstock_root/build_artifacts/webencodings_1694681268211/work
websocket-client @ file:///home/conda/feedstock_root/build_artifacts/websocket-client_1701630677416/work
Werkzeug==3.0.1
widgetsnbextension @ file:///home/conda/feedstock_root/build_artifacts/widgetsnbextension_1694598693908/work
wrapt==1.16.0
xxhash==3.4.1
xyzservices @ file:///home/conda/feedstock_root/build_artifacts/xyzservices_1698325309404/work
yachalk==0.1.5
yarl==1.9.4
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1695255097490/work
Note: you may need to restart the kernel to use updated packages.
mumianyuxin commented 4 months ago

Same error， any resolution？

jwjohns commented 3 months ago

same here as well.

guchuichui commented 1 month ago

I uninstalled and reinstalled the flash_attn module to solve the problem pip uninstall flash_attn pip install flash_attn

aulaywang commented 1 week ago

I met the same problem. I think it is caused because my CUDA version is 12.1. fkash-attn=2.5.6 does not work. Does anyone have the solution?