tianweiy / CenterPoint

MIT License
1.85k stars 450 forks source link

ValueError: Caught ValueError in DataLoader worker process 0. #69

Closed AI-liu closed 3 years ago

AI-liu commented 3 years ago

python ./tools/train.py configs/nusc/pp/nusc_centerpoint_pp_02voxel_two_pfn_10sweep.py No Tensorflow Deformable Convolution not built! No APEX! 2021-01-15 19:11:31,151 - INFO - Distributed training: False 2021-01-15 19:11:31,151 - INFO - torch.backends.cudnn.benchmark: False 2021-01-15 19:11:31,179 - INFO - Finish RPN Initialization 2021-01-15 19:11:31,179 - INFO - num_classes: [1, 2, 2, 1, 2, 2] Use HM Bias: -2.19 2021-01-15 19:11:31,208 - INFO - Finish CenterHead Initialization 2021-01-15 19:11:31,249 - INFO - {'car': 5, 'truck': 5, 'bus': 5, 'trailer': 5, 'construction_vehicle': 5, 'traffic_cone': 5, 'barrier': 5, 'motorcycle': 5, 'bicycle': 5, 'pedestrian': 5} 2021-01-15 19:11:31,249 - INFO - [-1] 2021-01-15 19:11:31,283 - INFO - load 3068 pedestrian database infos 2021-01-15 19:11:31,283 - INFO - load 4082 car database infos 2021-01-15 19:11:31,283 - INFO - load 773 traffic_cone database infos 2021-01-15 19:11:31,283 - INFO - load 147 bicycle database infos 2021-01-15 19:11:31,283 - INFO - load 1851 barrier database infos 2021-01-15 19:11:31,283 - INFO - load 451 truck database infos 2021-01-15 19:11:31,283 - INFO - load 337 bus database infos 2021-01-15 19:11:31,283 - INFO - load 174 construction_vehicle database infos 2021-01-15 19:11:31,284 - INFO - load 95 ignore database infos 2021-01-15 19:11:31,284 - INFO - load 179 motorcycle database infos 2021-01-15 19:11:31,284 - INFO - load 59 trailer database infos 2021-01-15 19:11:31,295 - INFO - After filter database: 2021-01-15 19:11:31,296 - INFO - load 2811 pedestrian database infos 2021-01-15 19:11:31,296 - INFO - load 3303 car database infos 2021-01-15 19:11:31,296 - INFO - load 639 traffic_cone database infos 2021-01-15 19:11:31,296 - INFO - load 136 bicycle database infos 2021-01-15 19:11:31,296 - INFO - load 1746 barrier database infos 2021-01-15 19:11:31,296 - INFO - load 412 truck database infos 2021-01-15 19:11:31,296 - INFO - load 309 bus database infos 2021-01-15 19:11:31,296 - INFO - load 161 construction_vehicle database infos 2021-01-15 19:11:31,296 - INFO - load 95 ignore database infos 2021-01-15 19:11:31,296 - INFO - load 149 motorcycle database infos 2021-01-15 19:11:31,296 - INFO - load 57 trailer database infos 10 2021-01-15 19:11:32,763 - INFO - model structure: PointPillars( (reader): PillarFeatureNet( (pfn_layers): ModuleList( (0): PFNLayer( (linear): Linear(in_features=10, out_features=32, bias=False) (norm): BatchNorm1d(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) ) (1): PFNLayer( (linear): Linear(in_features=64, out_features=64, bias=False) (norm): BatchNorm1d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) ) ) ) (backbone): PointPillarsScatter() (neck): RPN( (blocks): ModuleList( (0): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() ) (1): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() (13): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (14): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (15): ReLU() (16): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (17): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (18): ReLU() ) (2): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() (13): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (14): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (15): ReLU() (16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (17): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (18): ReLU() ) ) (deblocks): ModuleList( (0): Sequential( (0): Conv2d(64, 128, kernel_size=(2, 2), stride=(2, 2), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) (1): Sequential( (0): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) (2): Sequential( (0): ConvTranspose2d(256, 128, kernel_size=(2, 2), stride=(2, 2), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) ) ) (bbox_head): CenterHead( (crit): FastFocalLoss() (crit_reg): RegLoss() (shared_conv): Sequential( (0): Conv2d(384, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU(inplace=True) ) (tasks): ModuleList( (0): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (1): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (2): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (3): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (4): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (5): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) ) ) ) 2021-01-15 19:11:32,764 - INFO - Start running, host: ai@ai, work_dir: /home/ai/CenterPoint/work_dirs/nusc_centerpoint_pp_02voxel_two_pfn_10sweep 2021-01-15 19:11:32,764 - INFO - workflow: [('train', 1)], max: 20 epochs /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/150_construction_vehicle_14.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/12_bus_51.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/21_bus_49.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/12_motorcycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/12_motorcycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/248_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/13_bicycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/99_construction_vehicle_41.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/6_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/10_construction_vehicle_56.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/131_construction_vehicle_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/147_construction_vehicle_14.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/93_bus_19.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/288_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/23_motorcycle_5.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/11_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/303_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/307_motorcycle_16.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/22_bicycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/43_bicycle_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/261_bicycle_47.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/262_bicycle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/24_truck_58.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/99_construction_vehicle_41.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/6_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/10_construction_vehicle_56.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/131_construction_vehicle_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/147_construction_vehicle_14.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/113_construction_vehicle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/18_construction_vehicle_31.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/21_bus_49.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/228_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/93_bus_19.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/50_trailer_18.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/237_barrier_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/23_barrier_70.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/23_motorcycle_5.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/11_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/303_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/307_motorcycle_16.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/265_motorcycle_26.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/22_bicycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/261_bicycle_47.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/262_bicycle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_54.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/24_truck_58.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/99_construction_vehicle_41.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/12_bus_51.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/54_trailer_19.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/237_barrier_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/23_barrier_70.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/12_motorcycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/248_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/99_construction_vehicle_41.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/12_bus_51.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/54_trailer_19.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/237_barrier_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/23_barrier_70.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/12_motorcycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/248_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/15_truck_45.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/6_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/10_construction_vehicle_56.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/131_construction_vehicle_10.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/10_construction_vehicle_56.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/147_construction_vehicle_14.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/147_construction_vehicle_14.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/18_construction_vehicle_31.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/241_construction_vehicle_16.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/21_bus_49.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/21_bus_49.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/228_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/228_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/52_trailer_18.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/23_motorcycle_5.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/17_barrier_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/11_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/barrier/19_barrier_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/303_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/288_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/307_motorcycle_16.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/23_motorcycle_5.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/265_motorcycle_26.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/11_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/308_motorcycle_5.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/303_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/307_motorcycle_16.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/265_motorcycle_26.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/22_bicycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/22_bicycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/43_bicycle_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/43_bicycle_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/24_traffic_cone_66.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/261_bicycle_47.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/262_bicycle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_54.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/9_bicycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/31_truck_32.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/99_construction_vehicle_41.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/12_bus_51.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/12_bus_51.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/54_trailer_19.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/305_motorcycle_3.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/288_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/306_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/12_motorcycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/248_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/289_motorcycle_4.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/252_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/248_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/148_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/54_bicycle_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bicycle/138_bicycle_7.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/pedestrian/260_pedestrian_25.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/pedestrian/258_pedestrian_35.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/156_traffic_cone_1.bin /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/traffic_cone/125_traffic_cone_2.bin Traceback (most recent call last): File "./tools/train.py", line 137, in main() File "./tools/train.py", line 132, in main logger=logger, File "/home/ai/CenterPoint/det3d/torchie/apis/train.py", line 327, in train_detector trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank) File "/home/ai/CenterPoint/det3d/torchie/trainer/trainer.py", line 543, in run epoch_runner(data_loaders[i], self.epoch, **kwargs) File "/home/ai/CenterPoint/det3d/torchie/trainer/trainer.py", line 395, in train for i, data_batch in enumerate(data_loader): File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in next data = self._next_data() File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data return self._process_data(data) File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data data.reraise() File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) ValueError: Caught ValueError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop data = fetcher.fetch(index) File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ai/anaconda3/envs/centerpoint/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ai/CenterPoint/det3d/datasets/nuscenes/nuscenes.py", line 186, in getitem return self.get_sensor_data(idx) File "/home/ai/CenterPoint/det3d/datasets/nuscenes/nuscenes.py", line 181, in get_sensordata data, = self.pipeline(res, info) File "/home/ai/CenterPoint/det3d/datasets/pipelines/compose.py", line 25, in call res, info = t(res, info) File "/home/ai/CenterPoint/det3d/datasets/pipelines/preprocess.py", line 96, in call road_planes=None File "/home/ai/CenterPoint/det3d/core/sampler/sample_ops.py", line 217, in sample_all "points": np.concatenate(s_points_list, axis=0), File "<__array_function__ internals>", line 6, in concatenate ValueError: need at least one array to concatenate

tianweiy commented 3 years ago
 /media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/truck/99_truck_30.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/98_construction_vehicle_42.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/28_construction_vehicle_48.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/92_construction_vehicle_44.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/22_construction_vehicle_59.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/177_bus_6.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/construction_vehicle/150_construction_vehicle_14.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/trailer/113_trailer_25.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/bus/271_bus_9.bin
/media/ai/MyPassport1/nuScenes/gt_database_10sweeps_withvelo/motorcycle/10_motorcycle_9.bin

This means that these gt bin files are not found at the current path. Could you check to make sure if there are any files there?

HoAdrian commented 1 week ago

I can run dist_test.py perfectly fine. but the training script fails with the same error above at det3d/core/sampler/ssample_ops.py sample_all method on the line where s_points = np.fromfile(...). It turns out there is a runtime error in the try-except clause: RuntimeError: /io/build/temp.linux-x86_64-cpython-38/spconv/build/core_cc/src/csrc/sparse/convops/convops/ConvTunerSimple/ConvTunerSimple_tune_and_cache.cc(103) !all_profile_res.empty() assert faild. can't find suitable algorithm for 0