train on nuscenes KeyError: 'voxels'

751565516 commented 2 years ago

python ./tools/train.py ./configs/nusc/pp/nusc_centerpoint_pp_015voxel_two_pfn_10sweep.py

No Tensorflow 2022-07-20 10:19:14,336 INFO Distributed training: False 2022-07-20 10:19:14,336 INFO torch.backends.cudnn.benchmark: False 2022-07-20 10:19:14,449 INFO Backup files to : ./work_dirs/nusc_centerpoint_pp_015voxel_two_pfn_10sweep/backup 2022-07-20 10:19:14,469 INFO Finish RPN Initialization 2022-07-20 10:19:16,815 INFO num_classes: [1, 2, 2, 1, 2, 2] Use HM Bias: -2.19 2022-07-20 10:19:16,829 INFO Finish CenterHead Initialization 2022-07-20 10:19:23,147 INFO {'car': 5, 'truck': 5, 'bus': 5, 'trailer': 5, 'construction_vehicle': 5, 'traffic_cone': 5, 'barrier': 5, 'motorcycle': 5, 'bicycle': 5, 'pedestrian': 5} 2022-07-20 10:19:23,147 INFO [-1] 2022-07-20 10:19:24,614 INFO load 62964 traffic_cone database infos 2022-07-20 10:19:24,614 INFO load 65262 truck database infos 2022-07-20 10:19:24,614 INFO load 339949 car database infos 2022-07-20 10:19:24,614 INFO load 161928 pedestrian database infos 2022-07-20 10:19:24,614 INFO load 26297 ignore database infos 2022-07-20 10:19:24,614 INFO load 11050 construction_vehicle database infos 2022-07-20 10:19:24,614 INFO load 107507 barrier database infos 2022-07-20 10:19:24,614 INFO load 8846 motorcycle database infos 2022-07-20 10:19:24,614 INFO load 8185 bicycle database infos 2022-07-20 10:19:24,614 INFO load 12286 bus database infos 2022-07-20 10:19:24,614 INFO load 19202 trailer database infos 2022-07-20 10:19:25,249 INFO After filter database: 2022-07-20 10:19:25,250 INFO load 55823 traffic_cone database infos 2022-07-20 10:19:25,250 INFO load 60428 truck database infos 2022-07-20 10:19:25,250 INFO load 294575 car database infos 2022-07-20 10:19:25,250 INFO load 148872 pedestrian database infos 2022-07-20 10:19:25,250 INFO load 26297 ignore database infos 2022-07-20 10:19:25,250 INFO load 10591 construction_vehicle database infos 2022-07-20 10:19:25,250 INFO load 102093 barrier database infos 2022-07-20 10:19:25,250 INFO load 8055 motorcycle database infos 2022-07-20 10:19:25,250 INFO load 7533 bicycle database infos 2022-07-20 10:19:25,250 INFO load 11622 bus database infos 2022-07-20 10:19:25,250 INFO load 18104 trailer database infos 10 2022-07-20 10:19:25,275 INFO model structure: PointPillars( (reader): PillarFeatureNet( (pfn_layers): ModuleList( (0): PFNLayer( (linear): Linear(in_features=10, out_features=32, bias=False) (norm): BatchNorm1d(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) ) (1): PFNLayer( (linear): Linear(in_features=64, out_features=64, bias=False) (norm): BatchNorm1d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) ) ) ) (backbone): PointPillarsScatter() (neck): RPN( (blocks): ModuleList( (0): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() ) (1): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() (13): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (14): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (15): ReLU() (16): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (17): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (18): ReLU() ) (2): Sequential( (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0) (1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), bias=False) (2): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (3): ReLU() (4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (5): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (6): ReLU() (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (8): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (9): ReLU() (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (11): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (12): ReLU() (13): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (14): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (15): ReLU() (16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (17): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (18): ReLU() ) ) (deblocks): ModuleList( (0): Sequential( (0): Conv2d(64, 128, kernel_size=(2, 2), stride=(2, 2), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) (1): Sequential( (0): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) (2): Sequential( (0): ConvTranspose2d(256, 128, kernel_size=(2, 2), stride=(2, 2), bias=False) (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True) (2): ReLU() ) ) ) (bbox_head): CenterHead( (crit): FastFocalLoss() (crit_reg): RegLoss() (shared_conv): Sequential( (0): Conv2d(384, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU(inplace=True) ) (tasks): ModuleList( (0): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (1): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (2): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (3): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (4): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (5): SepHead( (reg): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (height): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (dim): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (rot): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (vel): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (hm): Sequential( (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) ) ) ) 2022-07-20 10:19:25,275 INFO Start running, host: zj@ubun, work_dir: /home/zj/PillarNet/work_dirs/nusc_centerpoint_pp_015voxel_two_pfn_10sweep 2022-07-20 10:19:25,275 INFO workflow: [('train', 1)], max: 20 epochs 2022-07-20 10:19:29,417 INFO finding looplift candidates 2022-07-20 10:19:29,435 INFO finding looplift candidates 2022-07-20 10:19:29,456 INFO finding looplift candidates 2022-07-20 10:19:29,466 INFO finding looplift candidates 2022-07-20 10:19:29,472 INFO finding looplift candidates 2022-07-20 10:19:29,478 INFO finding looplift candidates Traceback (most recent call last): File "./tools/train.py", line 172, in main() File "./tools/train.py", line 167, in main logger=logger, File "/home/zj/PillarNet/det3d/torchie/apis/train.py", line 326, in train_detector trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank) File "/home/zj/PillarNet/det3d/torchie/trainer/trainer.py", line 545, in run epoch_runner(data_loaders[i], self.epoch, kwargs) File "/home/zj/PillarNet/det3d/torchie/trainer/trainer.py", line 412, in train self.model, data_batch, train_mode=True, kwargs File "/home/zj/PillarNet/det3d/torchie/trainer/trainer.py", line 370, in batch_processor_inline losses = model(example, return_loss=True) File "/home/zj/anaconda3/envs/cp/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/home/zj/PillarNet/det3d/models/detectors/point_pillars.py", line 33, in forward voxels = example["voxels"] KeyError: 'voxels'

Can you tell me how to solve this problem? thank you！

tianweiy commented 2 years ago

Please describe your modifications. The original codebase + original config should work?

MakeNoah commented 2 years ago

I had the same problem. I tried to port pointpillar to this config (https://github.com/tianweiy/CenterPoint/blob/new_release/configs/waymo/voxelnet/waymo_centerpoint_voxelnet_ Three_sweeps_20e_iou_pred.py) to port pointpillar to this config (https://github.com/tianweiy/CenterPoint/blob/new_release/configs/waymo/ pp/waymo_centerpoint_pp_two_pfn_stride1_3x.py) to port the pointpillar. If you have already solved this problem, I would appreciate it if you could tell me how to solve it.

tianweiy commented 2 years ago

Hi @MakeNoah

Did you add a voxelization in your ported config ? It should look like the following. We don't have this in voxelnet as we use dynamic voxelization.

MakeNoah commented 2 years ago

I guess I forgot to change train_pipeline. This fixed the problem. Thank you so much for your help.

MakeNoah commented 2 years ago

If I may ask one more question, is it possible to change the .new_release to pointpillar? If you have a config that has been ported, I would appreciate it if you could add it to the repository.

tianweiy commented 2 years ago

I don't have at the moment. I think you can follow https://github.com/tianweiy/CenterPoint/blob/master/configs/waymo/pp/waymo_centerpoint_pp_two_pfn_stride1_3x.py

to change the assigner, test_cfg pc_range, voxel_size, and the voxel_generator

MakeNoah commented 2 years ago

Thank you for teaching me so kindly. I will try those changes.

tianweiy commented 2 years ago

yeah, once you finish this. feel free to paste it here and i can take a look

MakeNoah commented 2 years ago

@tianweiy Add pc_range and voxel_size to assigner, use_rotate_nms=True, use_multi_class_nms=False, cf_weight=2 to test_cfg, and voxel_generator to new_release's config. release config. I have the following settings, but I seem to get an error... Thank you for your help.

import itertools
import logging
from det3d.utils.config_tool import get_downsample_factor

tasks = [
    dict(num_class=3, class_names=['VEHICLE', 'PEDESTRIAN', 'CYCLIST']),
]

class_names = list(itertools.chain(*[t["class_names"] for t in tasks]))

# training and testing settings
target_assigner = dict(
    tasks=tasks,
)

# model settings
model = dict(
    type="PointPillars",
    pretrained=None,
    reader=dict(
        type="PillarFeatureNet",
        num_filters=[64, 64],
        num_input_features=6,
        with_distance=False,
        voxel_size=(0.32, 0.32, 6.0),
        pc_range=(-74.88, -74.88, -2, 74.88, 74.88, 4.0),
    ),
    backbone=dict(type="PointPillarsScatter", ds_factor=1),
    neck=dict(
        type="RPN",
        layer_nums=[3, 5, 5],
        ds_layer_strides=[1, 2, 2],
        ds_num_filters=[64, 128, 256],
        us_layer_strides=[1, 2, 4],
        us_num_filters=[128, 128, 128],
        num_input_features=64,
        logger=logging.getLogger("RPN"),
    ),
    bbox_head=dict(
        type="CenterHead",
        in_channels=128*3,
        tasks=tasks,
        dataset='waymo',
        weight=2,
        iou_weight=1,
        code_weights=[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],
        common_heads={'reg': (2, 2), 'height': (1, 2), 'dim':(3, 2), 'rot':(2, 2), 'iou': (1, 2)}, # (output_channel, num_conv)
    ),
)

assigner = dict(
    target_assigner=target_assigner,
    out_size_factor=get_downsample_factor(model),
    dense_reg=1,
    gaussian_overlap=0.1,
    max_objs=500,
    min_radius=2,
    pc_range=(-74.88, -74.88, -2, 74.88, 74.88, 4.0),
    voxel_size=(0.32, 0.32, 6.0),
)

train_cfg = dict(assigner=assigner)

test_cfg = dict(
    post_center_limit_range=[-80, -80, -10.0, 80, 80, 10.0],
    nms=dict(
        use_rotate_nms=True,
        use_multi_class_nms=False,
        nms_pre_max_size=4096,
        nms_post_max_size=500,
        nms_iou_threshold=0.7,
    ),
    score_threshold=0.1,
    pc_range=[-74.88, -74.88],
    out_size_factor=get_downsample_factor(model),
    voxel_size=[0.32, 0.32],
    cf_weight=2,
)

# dataset settings
dataset_type = "WaymoDataset"
nsweeps = 2
data_root = "data"

db_sampler = dict(
    type="GT-AUG",
    enable=False,
    db_info_path="data/dbinfos_train_2sweeps_withvelo.pkl",
    sample_groups=[
        dict(VEHICLE=15),
        dict(PEDESTRIAN=10),
        dict(CYCLIST=10),
    ],
    db_prep_steps=[
        dict(
            filter_by_min_num_points=dict(
                VEHICLE=5,
                PEDESTRIAN=5,
                CYCLIST=5,
            )
        ),
        dict(filter_by_difficulty=[-1],),
    ],
    global_random_rotation_range_per_object=[0, 0],
    rate=1.0,
) 

train_preprocessor = dict(
    mode="train",
    shuffle_points=True,
    global_rot_noise=[-0.78539816, 0.78539816],
    global_scale_noise=[0.95, 1.05],
    global_translate_std=0.5,
    db_sampler=db_sampler,
    class_names=class_names,
)

val_preprocessor = dict(
    mode="val",
    shuffle_points=False,
)

voxel_generator = dict(
    range=[-74.88, -74.88, -2, 74.88, 74.88, 4.0],
    voxel_size=[0.32, 0.32, 6.0],
    max_points_in_voxel=5,
    max_voxel_num=[250000, 400000], # we only use non-empty voxels. this will be much smaller than max_voxel_num
)

train_pipeline = [
    dict(type="LoadPointCloudFromFile", dataset=dataset_type),
    dict(type="LoadPointCloudAnnotations", with_bbox=True),
    dict(type="Preprocess", cfg=train_preprocessor),
    dict(type="Voxelization", cfg=voxel_generator),
    dict(type="AssignLabel", cfg=train_cfg["assigner"]),
    dict(type="Reformat"),
]
test_pipeline = [
    dict(type="LoadPointCloudFromFile", dataset=dataset_type),
    dict(type="LoadPointCloudAnnotations", with_bbox=True),
    dict(type="Preprocess", cfg=val_preprocessor),
    dict(type="Voxelization", cfg=voxel_generator),
    dict(type="AssignLabel", cfg=train_cfg["assigner"]),
    dict(type="Reformat"),
]

train_anno = "data/infos_train_02sweeps_filter_zero_gt.pkl"
val_anno = "data/infos_val_02sweeps_filter_zero_gt.pkl"
test_anno = "data/infos_test_02sweeps_filter_zero_gt.pkl"

data = dict(
    samples_per_gpu=2,
    workers_per_gpu=4,
    train=dict(
        type=dataset_type,
        root_path=data_root,
        info_path=train_anno,
        ann_file=train_anno,
        nsweeps=nsweeps,
        class_names=class_names,
        pipeline=train_pipeline,
    ),
    val=dict(
        type=dataset_type,
        root_path=data_root,
        info_path=val_anno,
        test_mode=True,
        ann_file=val_anno,
        nsweeps=nsweeps,
        class_names=class_names,
        pipeline=test_pipeline,
    ),
    test=dict(
        type=dataset_type,
        root_path=data_root,
        info_path=test_anno,
        ann_file=test_anno,
        nsweeps=nsweeps,
        class_names=class_names,
        pipeline=test_pipeline,
    ),
)

optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))

# optimizer
optimizer = dict(
    type="adam", amsgrad=0.0, wd=0.01, fixed_wd=True, moving_average=False,
)
lr_config = dict(
    type="one_cycle", lr_max=0.003, moms=[0.95, 0.85], div_factor=10.0, pct_start=0.4,
)

checkpoint_config = dict(interval=1)
# yapf:disable
log_config = dict(
    interval=5,
    hooks=[
        dict(type="TextLoggerHook"),
        # dict(type='TensorboardLoggerHook')
    ],
)
# yapf:enable
# runtime settings
total_epochs = 12
device_ids = range(8)
dist_params = dict(backend="nccl", init_method="env://")
log_level = "INFO"
work_dir = './work_dirs/{}/'.format(__file__[__file__.rfind('/') + 1:-3])
load_from = None 
resume_from = None  
workflow = [('train', 1)]

2022-09-26 19:42:53,288 - INFO - Distributed training: False
2022-09-26 19:42:53,288 - INFO - torch.backends.cudnn.benchmark: False
2022-09-26 19:42:53,416 - INFO - Finish RPN Initialization
2022-09-26 19:42:53,416 - INFO - num_classes: [3]
2022-09-26 19:42:53,432 - INFO - Finish CenterHead Initialization
2022-09-26 19:42:55,928 - INFO - {'VEHICLE': 5, 'PEDESTRIAN': 5, 'CYCLIST': 5}
2022-09-26 19:42:55,931 - INFO - [-1]
2022-09-26 19:43:04,700 - INFO - load 1087711 VEHICLE database infos
2022-09-26 19:43:04,700 - INFO - load 1018200 PEDESTRIAN database infos
2022-09-26 19:43:04,700 - INFO - load 49518 CYCLIST database infos
2022-09-26 19:43:07,827 - INFO - After filter database:
2022-09-26 19:43:07,828 - INFO - load 1052366 VEHICLE database infos
2022-09-26 19:43:07,828 - INFO - load 986944 PEDESTRIAN database infos
2022-09-26 19:43:07,828 - INFO - load 48619 CYCLIST database infos
2022-09-26 19:43:23,164 - INFO - model structure: PointPillars(
  (reader): PillarFeatureNet(
    (pfn_layers): ModuleList(
      (0): PFNLayer(
        (linear): Linear(in_features=11, out_features=32, bias=False)
        (norm): BatchNorm1d(32, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
      )
      (1): PFNLayer(
        (linear): Linear(in_features=64, out_features=64, bias=False)
        (norm): BatchNorm1d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
      )
    )
  )
  (backbone): PointPillarsScatter()
  (neck): RPN(
    (blocks): ModuleList(
      (0): Sequential(
        (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
        (1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), bias=False)
        (2): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (3): ReLU()
        (4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (5): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (6): ReLU()
        (7): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (8): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (9): ReLU()
        (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (11): BatchNorm2d(64, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (12): ReLU()
      )
      (1): Sequential(
        (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
        (1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), bias=False)
        (2): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (3): ReLU()
        (4): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (5): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (6): ReLU()
        (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (8): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (9): ReLU()
        (10): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (11): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (12): ReLU()
        (13): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (14): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (15): ReLU()
        (16): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (17): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (18): ReLU()
      )
      (2): Sequential(
        (0): ZeroPad2d(padding=(1, 1, 1, 1), value=0.0)
        (1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), bias=False)
        (2): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (3): ReLU()
        (4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (5): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (6): ReLU()
        (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (8): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (9): ReLU()
        (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (11): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (12): ReLU()
        (13): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (14): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (15): ReLU()
        (16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (17): BatchNorm2d(256, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (18): ReLU()
      )
    )
    (deblocks): ModuleList(
      (0): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (2): ReLU()
      )
      (1): Sequential(
        (0): ConvTranspose2d(128, 128, kernel_size=(2, 2), stride=(2, 2), bias=False)
        (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (2): ReLU()
      )
      (2): Sequential(
        (0): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(4, 4), bias=False)
        (1): BatchNorm2d(128, eps=0.001, momentum=0.01, affine=True, track_running_stats=True)
        (2): ReLU()
      )
    )
  )
  (bbox_head): CenterHead(
    (crit): FastFocalLoss()
    (crit_reg): RegLoss()
    (shared_conv): Sequential(
      (0): Conv2d(384, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU(inplace=True)
    )
    (tasks): ModuleList(
      (0): SepHead(
        (reg): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (height): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (dim): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (rot): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 2, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (iou): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
        (hm): Sequential(
          (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
          (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (2): ReLU()
          (3): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        )
      )
    )
  )
)
2022-09-26 19:43:23,165 - INFO - Start running
2022-09-26 19:43:23,166 - INFO - workflow: [('train', 1)], max: 12 epochs
No Tensorflow
Deformable Convolution not built!
True
Use HM Bias:  -2.19
Using 2 sweeps
Using 158081 Frames
2022-09-26 19:43:29,928 - INFO - finding looplift candidates
2022-09-26 19:43:29,956 - INFO - finding looplift candidates
2022-09-26 19:43:29,957 - INFO - finding looplift candidates
2022-09-26 19:43:29,991 - INFO - finding looplift candidates
Traceback (most recent call last):
  File "./tools/train.py", line 138, in <module>
    main()
  File "./tools/train.py", line 133, in main
    logger=logger,
  File "/home/user_name/CenterPoint/det3d/torchie/apis/train.py", line 326, in train_detector
    trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank)
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 542, in run
    epoch_runner(data_loaders[i], self.epoch, **kwargs)
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 409, in train
    self.model, data_batch, train_mode=True, **kwargs
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 367, in batch_processor_inline
    losses = model(example, return_loss=True)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/user_name/CenterPoint/det3d/models/detectors/point_pillars.py", line 52, in forward
    return self.bbox_head.loss(example, preds)
  File "/home/user_name/CenterPoint/det3d/models/bbox_heads/center_head.py", line 297, in loss
    preds_dict['hm'] = self._sigmoid(preds_dict['hm'])
TypeError: list indices must be integers or slices, not str

tianweiy commented 2 years ago

https://github.com/tianweiy/CenterPoint/blob/c4b4e0fc97ec6d42d6a31f559816d3305abdf750/det3d/models/detectors/point_pillars.py#L49

could you change the above line to be similar to voxelnet's

I forgot to fix pp implementation in this branch

MakeNoah commented 2 years ago

thx, I changed forward function of point_pillars.py to this.

 def forward(self, example, return_loss=True, **kwargs):
        voxels = example["voxels"]
        coordinates = example["coordinates"]
        num_points_in_voxel = example["num_points"]
        num_voxels = example["num_voxels"]

        batch_size = len(num_voxels)

        data = dict(
            features=voxels,
            num_voxels=num_points_in_voxel,
            coors=coordinates,
            batch_size=batch_size,
            input_shape=example["shape"][0],
        )

        x,_ = self.extract_feat(example)
        preds,_ = self.bbox_head(x)

        if return_loss:
            return self.bbox_head.loss(example, preds, self.test_cfg)
        else:
            return self.bbox_head.predict(example, preds, self.test_cfg)

Then the following error occurred

Traceback (most recent call last):
  File "./tools/train.py", line 138, in <module>
    main()
  File "./tools/train.py", line 133, in main
    logger=logger,
  File "/home/user_name/CenterPoint/det3d/torchie/apis/train.py", line 326, in train_detector
    trainer.run(data_loaders, cfg.workflow, cfg.total_epochs, local_rank=cfg.local_rank)
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 542, in run
    epoch_runner(data_loaders[i], self.epoch, **kwargs)
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 409, in train
    self.model, data_batch, train_mode=True, **kwargs
  File "/home/user_name/CenterPoint/det3d/torchie/trainer/trainer.py", line 367, in batch_processor_inline
    losses = model(example, return_loss=True)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/user_name/CenterPoint/det3d/models/detectors/point_pillars.py", line 48, in forward
    x,_ = self.extract_feat(example)
  File "/home/user_name/CenterPoint/det3d/models/detectors/point_pillars.py", line 23, in extract_feat
    data["features"], data["num_voxels"], data["coors"]
KeyError: 'features'

Do I need to change the extract_feat function as well?

tianweiy commented 2 years ago

use this

data = dict(
            features=voxels,
            num_voxels=num_points_in_voxel,
            coors=coordinates,
            batch_size=batch_size,
            input_shape=example["shape"][0],
        )

        x = self.extract_feat(data)
        preds, _ = self.bbox_head(x)

MakeNoah commented 2 years ago

After making the above changes, I was able to successfully start learning! Thank you so much!

MakeNoah commented 2 years ago

@tianweiy Hello. Thank you for everything you did for me

I have successfully implemented PointPillar based on CenterPoint++, but the inference speed seems to be lower than that of Voxelnet-base. In the paper, I heard that the main factor of speed-up is dynamic voxelization, but does it mean that it is not working effectively?

Here is the inference log of CenterPoint++(Voxelnet,voxelsize = [0.1, 0.1, 0.15])

2022-10-24 07:26:00,697 - INFO - Distributed testing: False
2022-10-24 07:26:00,698 - INFO - torch.backends.cudnn.benchmark: False
2022-10-24 07:26:00,890 - INFO - Finish RPN Initialization
2022-10-24 07:26:00,891 - INFO - num_classes: [3]
2022-10-24 07:26:00,906 - INFO - Finish CenterHead Initialization
2022-10-24 07:26:18,837 - INFO - work dir: work_dirs/WCVThree2twosweeps20eIOUpred/20221024_07-25-53/testset
No Tensorflow
Deformable Convolution not built!
Use HM Bias:  -2.19
Use Test Set
Using 2 sweeps
Using 39987 Frames
[                              ] 0/39987, elapsed: 0s, ETA:
[                              ] 1/39987, 0.9 task/s, elapsed: 1s, ETA: 43727s
[                              ] 2/39987, 1.7 task/s, elapsed: 1s, ETA: 23899s
...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ] 39986/39987, 6.0 task/s, elapsed: 6687s, ETA:     0s
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 39987/39987, 6.0 task/s, elapsed: 6687s, ETA:     0s
 Total time per frame:  0.16741466894270698

Here is the inference log for CenterPoint++(PointPillar,Voxelsize = [0.32,0.32,6.0])

2022-10-30 11:00:33,369 - INFO - Distributed testing: False
2022-10-30 11:00:33,370 - INFO - torch.backends.cudnn.benchmark: False
2022-10-30 11:00:33,441 - INFO - Finish RPN Initialization
2022-10-30 11:00:33,441 - INFO - num_classes: [3]
2022-10-30 11:00:33,453 - INFO - Finish CenterHead Initialization
2022-10-30 11:00:54,382 - INFO - work dir: work_dirs/WCPP_valtotest_ppbase/20221030_11-00-25/valtotestset
No Tensorflow
Deformable Convolution not built!
Use HM Bias:  -2.19
Use Val Set(current use testset)
Using 2 sweeps
Using 39987 Frames
[                              ] 0/39987, elapsed: 0s, ETA:
[                              ] 1/39987, 0.4 task/s, elapsed: 3s, ETA: 104662s
[                              ] 2/39987, 0.7 task/s, elapsed: 3s, ETA: 54558s
...
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ] 39986/39987, 5.5 task/s, elapsed: 7293s, ETA:     0s
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 39987/39987, 5.5 task/s, elapsed: 7293s, ETA:     0s
 Total time per frame:  0.18169845104825694

CenterPoint++(PointPillar,Voxelsize = [0.48,0.48,6.0]) gives [Total time per frame: 0.14853648522653407,FPS:6.7

Although the voxelsize modification improved the speed, it does not seem to have much effect.

GPU is a Tesla V100 SXM2.

tianweiy commented 2 years ago

first of all, we use https://github.com/tianweiy/CenterPoint/pull/157#issuecomment-881522643 for inference following the waymo challenge requirement.

The test.py will also do data loading which gives inaccurate time.

I heard that the main factor of speed-up is dynamic voxelization, but does it mean that it is not working effectively?

I don't understand your meaning. Currently, dynamic voxelization is only implemented for the voxelnet (not pp).

MakeNoah commented 2 years ago

I see...thank you very much. I apologize for the trouble I have caused you with my lack of understanding. I will try using #157 (comment) for inference. Regarding the reimplementation of dynamic voxelization using GPUs to speed up the process, I understand that it is only for Voxelnet. Is it correct to say that a similar implementation can be applied to PointPillar to reduce the loss of quantization information and still speed up the process?

tianweiy / CenterPoint

train on nuscenes KeyError: 'voxels' #339