search

iot-salzburg / gpu-jupyter

GPU-Jupyter: Leverage the flexibility of Jupyterlab through the power of your NVIDIA GPU to run your code from Tensorflow and Pytorch in collaborative notebooks on the GPU.
Apache License 2.0
708 stars 235 forks source link

cuda.h not found #86

Closed ScarWar closed 1 year ago

ScarWar commented 2 years ago

Hello, I think we are missing an env. variable in the Dockerfile on the location of the cuda headers.

I ran the docker on my university GPU cluster using udocker , and I'm using a library called KeOps which uses cuda intensively. But every time I tried to import keops in my project I had a problem with cuda.h file not being found, although when I ran torch.cuda.is_avaliable() I was getting True. I was searching for a solution for a long time, and after looking at the keops-full dockerfile I found a tiny but essential env. variable CUDA_PATH, which is set to /opt/conda/, which indeed contains cuda.h file.

After setting in ~/.bashrc the env. variable export CUDA_PATH=/opt/conda/ and restarting the docker I ran the import again and found success.

(base) root@ise-pheno-01:~# python -c "from pykeops.torch import LazyTensor""
[KeOps] Compiling cuda jit compiler engine ... OK
[pyKeOps] Compiling nvrtc binder for python ... OK

I think that this env. is missing and must be added to the dockerfile

mathematicalmichael commented 2 years ago

oh interesting. Is that env variable specific to your library? If so I'd argue that your use case requires modifying the dockerfile to install your binaries + set up the env variables they need.

If it's something required by tflow/torch/other-libs, then I can see an argument for including it by default.

ScarWar commented 2 years ago

Thanks for the quick repaly

No, the env. var is not specific to any library, but rather it is related to the CUDA SDK location. AFAIK, tf and torch come with their own cuda.h, but many other libraries, including Nvidia's, need it set correctly (quick search results).

mathematicalmichael commented 1 year ago

Ah I see. Well then I think we should add this! Thanks for bringing it to our attention.

Seems like a simple addition to the Dockerfile. Would you like to open a PR or should I do so on your behalf?

ChristophSchranz commented 1 year ago

Hi @ScarWar this is definitely important and will be part of the next release. Thank you!

ChristophSchranz commented 1 year ago

Solved with PR #87