search

V2AI / Det3D

World's first general purpose 3D object detection codebse.
https://arxiv.org/abs/1908.09492
Apache License 2.0
1.5k stars 298 forks source link

Training nuscence dataset error #95

Closed jinglin80 closed 4 years ago

jinglin80 commented 4 years ago

While following the instruction in https://github.com/poodarchu/Det3D/blob/master/GETTING_STARTED.md to train with command: ./tools/scripts/train.sh nuScenes, I got the following error.

AttributeError: Caught AttributeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/usr/local/Det3D/det3d/datasets/nuscenes/nuscenes.py", line 177, in getitem return self.get_sensor_data(idx) File "/usr/local/Det3D/det3d/datasets/nuscenes/nuscenes.py", line 172, in get_sensordata data, = self.pipeline(res, info) File "/usr/local/Det3D/det3d/datasets/pipelines/compose.py", line 23, in call res, info = t(res, info) File "/usr/local/Det3D/det3d/datasets/pipelines/loading.py", line 106, in call sweep_times_list = [np.zeros((points.shape[0], 1))] AttributeError: 'NoneType' object has no attribute 'shape'

Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 253, in main() File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 249, in main cmd=cmd) subprocess.CalledProcessError: Command '['/usr/bin/python3', '-u', '/shares/xionggroup/Det3D/tools/train.py', '--local_rank=1', '/shares/xionggroup/Det3D/examples/cbgs/configs/nusc_all_vfev3_spmiddleresnetfhd_rpn2_mghead_syncbn.py', '--work_dir=/shares/xionggroup/Det3D/Outputs/Det3D_Outputs/']' returned non-zero exit status 1.

poodarchu commented 4 years ago

check line 106 in /usr/local/Det3D/det3d/datasets/pipelines/loading.py, it seems that data didn't read correctly.

Asfiya-Baig commented 4 years ago

I am facing the same issue, but for validation part. The training happens just fine. Another point to note is that I am performing the training & validation on just 'v1.0-trainval01_lidar_blobs', which is just a part of the entire dataset. Could this be an issue?

I am confused because my training worked fine, and the error is thrown just during validation.

Upon checking which lidar file throws the error, I see that it doesn't exist. So is the create_data.py creating data for the entire dataset instead of the part of the dataset that I have downloaded.