search

yahoo / photo-background-generation

Apache License 2.0
39 stars 3 forks source link

cuDNN error: CUDNN_STATUS_EXECUTION_FAILED #4

Open carlos-havier opened 3 hours ago

carlos-havier commented 3 hours ago

Hi,

Very cool project!

I installed in clean Anaconda environments with the following configs:

And then running:

pip install -r requirements.txt

And getting from python -c "import torch; print(torch.cuda.is_available()); print(torch.version)":

True 2.2.2+cu121

In both, I run the test code at Huggingface (https://huggingface.co/yahoo-inc/photo-background-generation).

The step of the code to get the foreground mask:

fg_mask = remover.process(img, type='map') # default setting - transparent background

Gives the following error:

File ~/anaconda3/envs/pbg/lib/python3.9/site-packages/torch/nn/functional.py:2482, in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps) 2479 if training: 2480 _verify_batch_size(input.size()) -> 2482 return torch.batch_norm( 2483 input, weight, bias, running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled 2484 )

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

I discard a HW or library problem, given that I have many other environments running with CUDA 11.3, 11.8 and 12.1 for AI without issue.

Help figuring this out would be much appreciated

Thanks! :)

WinXsp commented 3 hours ago

Is the appropriate version of CUDNN not installed.

Please refer to my installation command:

conda install pytorch torchvision torchaudio pytorch-cuda=12.4 cudnn=8.9.2.26=cuda12_0 -c pytorch-nightly -c nvidia -y --force-reinstall