search

facebookresearch / xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.
https://facebookresearch.github.io/xformers/
Other
8.33k stars 585 forks source link

linux install fatal error: cusolverDn.h #607

Open mmann1123 opened 1 year ago

mmann1123 commented 1 year ago

Issue

Interestingly the conda build installs fine - but does not work. Meanwhile the pip build fails with the following:

  /home/name/miniconda3/envs/sd_au1111_v2/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:10:10: fatal error: cusolverDn.h: No such file or directory

I am finding exceedingly few examples of how to resolve this online and wondering if you have any ideas.

Install

mamba create -n sd_au1111_v2 python=3.10.6 -y
mamba activate sd_au1111_v2
mamba install pytorch torchvision pytorch-lightning torchaudio pytorch-cuda=11.6  -c pytorch -c nvidia  -y
# mamba install xformers -c xformers/label/dev
pip install ninja
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers

System

Linux pop-os 6.0.6-76060006-generic #202210290932~1669062050~22.04~d94609a SMP PREEMPT_DYNAMIC Mon N x86_64 x86_64 x86_64 GNU/Linux

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Oct_12_20:09:46_PDT_2020
Cuda compilation tools, release 11.1, V11.1.105
Build cuda_11.1.TC455_06.29190527_0
AbdBarho commented 1 year ago

@mmann1123 which torch version is installed? I assume 1.13 right?

For the pip install: could you try and update nvcc? you have Cuda compilation tools, release 11.1 and pytorch-cuda=11.6.

mmann1123 commented 1 year ago

Torch is 1.13, I can upgrade to cuda 11.2.

Seems odd to be missing a header file though right?

mmann1123 commented 1 year ago

Updating cuda seems to have resolved this.

korzen commented 1 year ago

I am getting this error on PyTorch 1.13.1 py3.8_cuda11.6_cudnn8.3.2_0 and: nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Tue_Mar__8_18:18:20_PST_2022 Cuda compilation tools, release 11.6, V11.6.124 Build cuda_11.6.r11.6/compiler.31057947_0

artkorenev commented 1 year ago

I ran into the same problem lately when I was trying to switch to pytorch-nightly build installed via conda. While pytorch itself installs correctly and works fine, I couldn't build xformers anymore.

It seems like the wrong version of libcusolver is installed within the process of packages resolution and there is no way to force conda to install another version of it without causing conflicts (which would not be resolved and thus no pytorch will be installed).

What worked for me though is installing nightly pytorch from pip in a new environment from scratch: pip3 install --pre torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu116