search

V2AI / Det3D

World's first general purpose 3D object detection codebse.
https://arxiv.org/abs/1908.09492
Apache License 2.0
1.48k stars 299 forks source link

How to test nuscenes #140

Closed liangchunlan closed 2 years ago

liangchunlan commented 3 years ago

KITTI:

When I train KITTI:

(base) ubuntu@ubuntu-System-Product-Name:~/mycode/Det3D$ python tools/create_data.py kitti_data_prep --root_path=/media/ubuntu/nuscenes/dataset/KITTI

modify train.sh , set work_ DIR python -m torch.distributed.launch --nproc_per_node=1 ./tools/train.py ./examples/point_pillars/configs/kitti_point_pillars_mghead_syncbn.py --work_dir=$PP_WORK_DIR

Run in command (base) ubuntu@ubuntu-System-Product-Name:~/mycode/Det3D$./tools/scripts/train.sh KITTI

result:

Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:90.63, 88.98, 87.78 bev AP:89.95, 87.41, 85.86 3d AP:87.10, 76.99, 74.99 aos AP:90.30, 88.37, 86.94 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:90.63, 88.98, 87.78 bev AP:90.70, 89.83, 89.21 3d AP:90.70, 89.76, 89.05 aos AP:90.30, 88.37, 86.94

Evaluation coco: car coco AP@0.50:0.05:0.95: bbox AP:71.73, 68.14, 66.06 bev AP:70.16, 66.44, 65.15 3d AP:60.23, 55.39, 53.77 aos AP:70.96, 67.28, 65.05

INFO - Epoch(val) [100][1257]
INFO - task : ['Car']

the task in point_pillars/configs/kitti_point_pillars_mghead_syncbn.py is： tasks = [ dict(num_class=1, class_names=["Car",],), ]； I wonder if this can present more categories， likes this： tasks = [ dict(num_class=1, class_names=["Car"]), dict(num_class=1, class_names=["Pedestrian"]), dict(num_class=1, class_names=["Cyclist"]), ]；

But it's dosen't work to change it like this. I want to know if it's really dosen't work or if I haven't changed it in other places.

nuscenes:

(base) ubuntu@ubuntu-System-Product-Name:~/mycode/Det3D$ python tools/create_data.py nuscenes_data_prep --root_path=/media/ubuntu/nuscenes/dataset/v1.0-mini --version="v1.0-mini" --nsweeps=10

modify train.sh , set work_ DIR python -m torch.distributed.launch --nproc_per_node=1 ./tools/train.py ./examples/point_pillars/configs/nusc_all_point_pillars_mghead_syncbn.py --work_dir=$PP_WORK_DIR Run in command (base) ubuntu@ubuntu-System-Product-Name:~/mycode/Det3D$ ./tools/scripts/train.sh NuScenes

There won't be a direct result like Kitti. Is there a problem with my running（./examples/point_pillars/configs/nusc_all_point_pillars_mghead_syncbn.py XXX has been modified.）

the result is: 2021-01-15 15:10:00,333 - INFO - Epoch [20/20][405/408] lr: 0.00000, eta: 0:00:01, time: 0.319, data_time: 0.112, transfer_time: 0.034, forward_time: 0.059, loss_parse_time: 0.000 memory: 4255, 2021-01-15 15:10:00,333 - INFO - task : ['car'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 17.2000, num_neg: 32730.0000 2021-01-15 15:10:00,333 - INFO - task : ['truck', 'construction_vehicle'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 29.4000, num_neg: 65442.6000 2021-01-15 15:10:00,334 - INFO - task : ['bus', 'trailer'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 32.4000, num_neg: 65465.2000 2021-01-15 15:10:00,334 - INFO - task : ['barrier'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 14.0000, num_neg: 32750.0000 2021-01-15 15:10:00,334 - INFO - task : ['motorcycle', 'bicycle'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 11.0000, num_neg: 65519.6000 2021-01-15 15:10:00,334 - INFO - task : ['pedestrian', 'traffic_cone'], loss: nan, cls_pos_loss: nan, cls_neg_loss: nan, dir_loss_reduced: 0.0000, cls_loss_reduced: nan, loc_loss_reduced: nan, loc_loss_elem: ['nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan'], num_pos: 21.6000, num_neg: 65511.0000

According to the: To test a model with ./tools/scripts/test.sh, run:

./tools/scripts/test.sh CONFIG_PATH WORK_DIR CHECKPOINT

Where CHECKPOINT represents the path of model file.

My command is like this. I want to ask if there are problems here. I have problems here: (base) ubuntu@ubuntu-System-Product-Name:~/mycode/Det3D$./tools/scripts/test.sh ./examples/point_pillars/configs/nusc_all_point_pillars_mghead_syncbn.py /media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs /media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs/PointPillars_NuScenes_20210115-140428/latest.pth

CONFIG_PATH=./examples/point_pillars/configs/nusc_all_point_pillars_mghead_syncbn.py WORK_DIR=/media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs CHECKPOINT=/media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs/PointPillars_NuScenes_20210115-140428/latest.pth

result:

2021-01-15 19:24:48,496 - INFO - Distributed testing: False 2021-01-15 19:24:48,496 - INFO - torch.backends.cudnn.benchmark: False 2021-01-15 19:24:48,542 - INFO - Finish RPN Initialization 2021-01-15 19:24:48,542 - INFO - num_classes: [1, 2, 2, 1, 2, 2], num_preds: [20, 40, 40, 20, 40, 40], num_dirs: [] 2021-01-15 19:24:48,544 - INFO - Finish MultiGroupHead Initialization 2021-01-15 19:24:49,741 - INFO - work dir: /media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 81/81, 13.0 task/s, elapsed: 6s, ETA: 0s Loading NuScenes tables for version v1.0-mini... 23 category, 8 attribute, 4 visibility, 911 instance, 12 sensor, 120 calibrated_sensor, 31206 ego_pose, 8 log, 10 scene, 404 sample, 31206 sample_data, 18538 sample_annotation, 4 map, Done loading in 0.393 seconds.

Reverse indexing ... Done reverse indexing in 0.1 seconds.

Finish generate predictions for testset, save to /media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs/infos_val_10sweeps_withvelo.json Initializing nuScenes detection evaluation Loaded results from /media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs/infos_val_10sweeps_withvelo.json. Found detections for 81 samples. Loading annotations for mini_val split from nuScenes version: v1.0-mini 100%|██████████████████████████████████████████| 81/81 [00:00<00:00, 365.69it/s] Loaded ground truth annotations for 81 samples. Filtering predictions Traceback (most recent call last): File "./tools/dist_test.py", line 196, in main() File "./tools/dist_test.py", line 166, in main resultdict, = dataset.evaluation(predictions, output_dir=args.work_dir) File "/home/ubuntu/mycode/Det3D/det3d/datasets/nuscenes/nuscenes.py", line 286, in evaluation eval_main( File "/home/ubuntu/mycode/Det3D/det3d/datasets/nuscenes/nusc_common.py", line 723, in eval_main nusc_eval = NuScenesEval( File "/home/ubuntu/mycode/nuscenes-devkit/python-sdk/nuscenes/eval/detection/evaluate.py", line 116, in init self.pred_boxes = filter_eval_boxes(nusc, self.pred_boxes, self.cfg.class_range, verbose=verbose) File "/home/ubuntu/mycode/nuscenes-devkit/python-sdk/nuscenes/eval/common/loaders.py", line 219, in filter_eval_boxes class_field = _get_box_class_field(eval_boxes) File "/home/ubuntu/mycode/nuscenes-devkit/python-sdk/nuscenes/eval/common/loaders.py", line 283, in _get_box_class_field raise Exception('Error: Invalid box type: %s' % box) Exception: Error: Invalid box type: None Traceback (most recent call last): File "/home/ubuntu/anaconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/ubuntu/anaconda3/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/ubuntu/anaconda3/lib/python3.8/site-packages/torch/distributed/launch.py", line 263, in main() File "/home/ubuntu/anaconda3/lib/python3.8/site-packages/torch/distributed/launch.py", line 258, in main raise subprocess.CalledProcessError(returncode=process.returncode, subprocess.CalledProcessError: Command '['/home/ubuntu/anaconda3/bin/python', '-u', './tools/dist_test.py', '--local_rank=0', './examples/point_pillars/configs/nusc_all_point_pillars_mghead_syncbn.py', '--work_dir=/media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs', '--checkpoint=/media/ubuntu/nuscenes/dataset/v1.0-mini/Det3D_Outputs/PointPillars_NuScenes_20210115-140428/latest.pth']' returned non-zero exit status 1.

poodarchu commented 3 years ago

Did you use the latest version?