SamsungLabs / imvoxelnet

[WACV2022] ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection
MIT License
283 stars 29 forks source link

run error about the Nuscenes dataset. #17

Closed rockywind closed 3 years ago

rockywind commented 3 years ago

Thank you for your help on the previous issue. I run the test script( test.py) in the Nuscenes data, the error is below. But, it's normal when I run the test script in the kitti dataset. `root@39370106c70e:/workspace/imvoxel# sh eval.sh [ ] 0/6019, elapsed: 0s, ETA:ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm). Traceback (most recent call last): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 986, in _try_get_data data = self._data_queue.get(timeout=timeout) File "/opt/conda/lib/python3.7/multiprocessing/queues.py", line 104, in get if not self._poll(timeout): File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 257, in poll return self._poll(timeout) File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 414, in _poll r = wait([self], timeout) File "/opt/conda/lib/python3.7/multiprocessing/connection.py", line 920, in wait ready = selector.select(timeout) File "/opt/conda/lib/python3.7/selectors.py", line 415, in select fd_event_list = self._selector.poll(timeout) File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler _error_if_any_worker_fails() RuntimeError: DataLoader worker (pid 5384) is killed by signal: Bus error. It is possible that dataloader's workers are out of shared memory. Please try to raise your shared memory limit.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "tools/test.py", line 155, in main() File "tools/test.py", line 131, in main outputs = single_gpu_test(model, data_loader, args.show, args.show_dir) File "/workspace/imvoxel/mmdet3d/apis/test.py", line 27, in single_gpu_test for i, data in enumerate(data_loader): File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in next data = self._next_data() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1182, in _next_data idx, data = self._get_data() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1148, in _get_data success, data = self._try_get_data() File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 999, in _try_get_data raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e RuntimeError: DataLoader worker (pid(s) 5384) exited unexpectedly `

filaPro commented 3 years ago

Hi @rockywind ,

Most likely this bug is unrelated to imvoxelnet and for sure can be googled as a pytorch issue. Are you using docker? Try increasinr your shared memory by --default-shm-size="16gb".

rockywind commented 3 years ago

Thank you for your help on the previous issue. Yes, I am using docker. It works for me. sudo docker run --shm-size=16G -v /newnfs/zzwu/08_3d_code/imvoxelnet05/:/workspace/imvoxel -it imvoxelnet_save