facebookresearch / mvp

Training and Evaluation Code for "Mixture of Volumetric Primitives for Efficient Neural Rendering"
Other
191 stars 16 forks source link

build success, but cannot run #8

Closed AN-ZE closed 1 year ago

AN-ZE commented 1 year ago
Traceback (most recent call last):
  File "render.py", line 118, in <module>
    output, _ = ae(
  File "/home/an/anaconda3/envs/py38-t19/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1186, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/an/an_project/mvp/models/volumetric.py", line 286, in forward
    rayrgba, rmlosses = self.raymarcher(raypos, raydir, tminmax,
  File "/home/an/anaconda3/envs/py38-t19/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1186, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/an/an_project/mvp/models/raymarchers/mvpraymarcher.py", line 32, in forward
    rayrgba = mvpraymarch(raypos, raydir, dt, tminmax,
  File "/home/an/an_project/mvp/extensions/mvpraymarch/mvpraymarch.py", line 273, in mvpraymarch
    out = MVPRaymarch.apply(raypos, raydir, stepsize, tminmax,
  File "/home/an/an_project/mvp/extensions/mvpraymarch/mvpraymarch.py", line 119, in forward
    sortedobjid, nodechildren, nodeaabb = build_accel(primtransfin,
  File "/home/an/an_project/mvp/extensions/mvpraymarch/mvpraymarch.py", line 44, in build_accel
    sortedobjid = (torch.arange(N*K, dtype=torch.int32, device=dev) % K).view(N, K)
RuntimeError: CUDA error: no kernel image is available for execution on the device

use: python 3.8.13 pytorch 1.13.0a0+git4503c45 cuda 11.3.0 gcc version 8.4.0


Hi, are you having similar issues?
pytorch is working normally.

stephenlombardi commented 1 year ago

Hi, I haven't seen this problem. Can you try printing "dev" to see what the value is? This doesn't seem to be an issue with the extension's C++ or CUDA code since it hits this error before reaching those calls.

shuishiwojiade commented 1 year ago

Hi, I haven't seen this problem. Can you try printing "dev" to see what the value is? This doesn't seem to be an issue with the extension's C++ or CUDA code since it hits this error before reaching those calls.

I meet this error too. The Value of dev is: cuda:0.

my envirionment is: python 3.9.0 pytorch 1.11.0=py3.9_cuda11.3_cudnn8.2.0_0 cuda 11.3 gcc version 9.3.0 ubuntu 20.04

stephenlombardi commented 1 year ago

Could you try re-running the script with CUDA_LAUNCH_BLOCKING=1 environment variable set? It's possible the exception is triggering from another line.

This seems like it might also be related to this issue: https://github.com/NVlabs/stylegan2-ada-pytorch/issues/6 and there are some suggestions at the bottom you can try. One thing that might be worth trying is to use CUDA toolkit 11.1 and recompiling the extensions. Can you also let me know what GPU you are using and maybe run the environment collection script from this issue and post the results?

You can also try the suggestions from here: https://github.com/NVIDIA/apex/issues/605 and use the TORCH_CUDA_ARCH_LIST environment variable before building the extension.

stephenlombardi commented 1 year ago

Also, take a look at this file: https://github.com/facebookresearch/mvp/blob/main/extensions/mvpraymarch/setup.py

You may need to modify the -arch=sm_70 so that it matches your GPU.

AN-ZE commented 1 year ago

Also, take a look at this file: https://github.com/facebookresearch/mvp/blob/main/extensions/mvpraymarch/setup.py

You may need to modify the -arch=sm_70 so that it matches your GPU.

Thank you. The problem is solved, the problem really lies here, just comment out the error location when build. my GPU GTX Titan X, -arch=sm_50 Will the MAP training data be released?

stephenlombardi commented 1 year ago

The data is released: https://github.com/facebookresearch/multiface

I'll be working to update the repo with a dataset class for this data

shuishiwojiade commented 1 year ago

The data is released: https://github.com/facebookresearch/multiface

I'll be working to update the repo with a dataset class for this data

Multiface dataset has been used in "Mixture of Volumetric Primitives for Efficient Neural Rendering". The "mvp" onfig file needs the path to the background image. But I can't find the background image in multiface dataset. The code in config file is: bgpath = os.path.join(imagepathbase, 'bg', image', 'cam{cam}', 'image0000.png').

stephenlombardi commented 1 year ago

Please see this issue for background images: https://github.com/facebookresearch/mvp/issues/9