search

Samiepapa / pifuhd

High-Resolution 3D Human Digitization from A Single Image.
Other
0 stars 0 forks source link

RuntimeError: CUDA error: no kernel image is available for execution on the device #2

Open Samiepapa opened 2 years ago

Samiepapa commented 2 years ago 
$ sh ./scripts/demo.sh
Resuming from  ./checkpoints/pifuhd.pt
/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/cuda/__init__.py:143: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
Warning: opt is overwritten.
test data size:  1
initialize network with normal
initialize network with normal
generate mesh (test) ...
  0%|                                                                                                                                                                                                                  | 0/1 [00:00<?, ?it/s]./results/pifuhd_final/recon/result_test_512.obj
  0%|                                                                                                                                                                                                                  | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/yongilcho/anaconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/yongilcho/anaconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/yongilcho/reposit/pifuhd/apps/simple_test.py", line 30, in <module>
    reconWrapper(cmd, args.use_rect)
  File "/home/yongilcho/reposit/pifuhd/apps/recon.py", line 220, in reconWrapper
    recon(opt, use_rect)
  File "/home/yongilcho/reposit/pifuhd/apps/recon.py", line 210, in recon
    gen_mesh(opt.resolution, netMR, cuda, test_data, save_path, components=opt.use_compose)
  File "/home/yongilcho/reposit/pifuhd/apps/recon.py", line 38, in gen_mesh
    net.filter_global(image_tensor_global)
  File "/home/yongilcho/reposit/pifuhd/lib/model/HGPIFuMRNet.py", line 83, in filter_global
    self.netG.filter(images)
  File "/home/yongilcho/reposit/pifuhd/lib/model/HGPIFuNetwNML.py", line 122, in filter
    self.nmlF = self.netF.forward(images).detach()
  File "/home/yongilcho/reposit/pifuhd/lib/networks.py", line 163, in forward
    return self.model(input)
  File "/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/nn/modules/padding.py", line 174, in forward
    return F.pad(input, self.padding, 'reflect')
  File "/home/yongilcho/.virtualenvs/pifuhd/lib/python3.8/site-packages/torch/nn/functional.py", line 4189, in _pad
    return torch._C._nn.reflection_pad2d(input, pad)
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
freeglut (foo): failed to open display ''
Samiepapa commented 2 years ago

1) CUDA 버전 확인

$ nvidia-smi
Mon Feb  7 19:52:48 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.46       Driver Version: 495.46       CUDA Version: 11.5     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   22C    P8    22W / 350W |      5MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce ...  Off  | 00000000:2E:00.0 Off |                  N/A |
|  0%   23C    P8    17W / 350W |     71MiB / 24265MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA GeForce ...  Off  | 00000000:41:00.0 Off |                  N/A |
|  0%   23C    P8    26W / 350W |      5MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  NVIDIA GeForce ...  Off  | 00000000:61:00.0 Off |                  N/A |
|  0%   23C    P8    28W / 350W |      5MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2668      G   /usr/lib/xorg/Xorg                  4MiB |
|    1   N/A  N/A      2668      G   /usr/lib/xorg/Xorg                 55MiB |
|    1   N/A  N/A      2978      G   /usr/bin/gnome-shell               14MiB |
|    2   N/A  N/A      2668      G   /usr/lib/xorg/Xorg                  4MiB |
|    3   N/A  N/A      2668      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

2) CUDA 11.3 버전에 맞는 stable한 pytorch 버전 설치하기 (https://pytorch.org/get-started/locally/ 참고)

pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html