search

google / lasr

Code for "LASR: Learning Articulated Shape Reconstruction from a Monocular Video". CVPR 2021.
https://lasr-google.github.io/
Apache License 2.0
170 stars 17 forks source link

RuntimeError: CUDA error: invalid device ordinal #6

Closed Kana-alt closed 3 years ago

Kana-alt commented 3 years ago

I used the docker to build the environment. I prepared the DAVIS data and tried to run Optimize on camel observations.

Then, I got a CUDA error and the execution did not proceed.

Can you tell me the cause?

docker run -v $(pwd):/lasr --gpus all lasr bash -c 'cd lasr; source activate lasr; bash scripts/template.sh camel' Jitting Chamfer 3D Jitting Chamfer 3D Loaded JIT 3D CUDA chamfer distance Loaded JIT 3D CUDA chamfer distance Traceback (most recent call last): File "optimize.py", line 59, in app.run(main) File "/anaconda3/envs/lasr/lib/python3.8/site-packages/absl/app.py", line 303, in run _run_main(main, args) File "/anaconda3/envs/lasr/lib/python3.8/site-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "optimize.py", line 40, in main torch.cuda.set_device(opts.local_rank) File "/anaconda3/envs/lasr/lib/python3.8/site-packages/torch/cuda/init.py", line 263, in set_device torch._C._cuda_setDevice(device) RuntimeError: CUDA error: invalid device ordinal

gengshan-y commented 3 years ago

There seems to be a mismatch between number of gpus you have and requested here.

If there is less than two gpus, you could modify the script to use one gpu and larger batch size, which may take longer to train.

Kana-alt commented 3 years ago

thank you.