thangvubk / SoftGroup

[CVPR 2022 Oral] SoftGroup for Instance Segmentation on 3D Point Clouds
MIT License
346 stars 81 forks source link

Can't reproduce performance stated in paper #165

Closed Christinepan881 closed 1 year ago

Christinepan881 commented 1 year ago

Hi, thanks for your great work!

However, I cannot reproduce the performance stated in the paper on S3DIS dataset. Do you have any ideas about such results? I have also tried to reverse the diff but the results are worse....

### Here are my results on S3DIS: AP: 0.471 AP50: 0.6 AP25: 0.701 mIoU: 64.1 Acc: 88.0 Offset MAE: 2.529

image

### My training steps:

  1. 20 epochs for finetuning the pretrained HAIS backbone on S3DIS.
  2. 20 epochs for training the model from frozen backbone.

### Here is my training config: 2023-02-16 21:40:51,875 - INFO - Config: model: channels: 32 num_blocks: 7 semantic_classes: 13 instance_classes: 13 sem2ins_classes: [0, 1] semantic_only: False ignore_label: -100 grouping_cfg: score_thr: 0.2 radius: 0.04 mean_active: 300 class_numpoint_mean: [34229, 39796, 12210, 7457, 5439, 10225, 6016, 1724, 5092, 7424, 5279, 6189, 1823] npoint_thr: 0.05 # absolute if class_numpoint == -1, relative if class_numpoint != -1 ignore_classes: [0, 1] instance_voxel_cfg: scale: 50 spatial_shape: 20 train_cfg: max_proposal_num: 200 pos_iou_thr: 0.5 test_cfg: x4_split: False #True cls_score_thr: 0.001 mask_score_thr: -0.5 min_npoint: 100 eval_tasks: ['semantic', 'instance'] fixed_modules: ['input_conv', 'unet', 'output_layer', 'semantic_linear', 'offset_linear'] data: train: type: 's3dis' data_root: '/workplace/dataset/s3dis/preprocess' prefix: ['Area_1', 'Area_2', 'Area_3', 'Area_4', 'Area_6'] suffix: '_inst_nostuff.pth' repeat: 20 training: True voxel_cfg: scale: 50 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000 x4_split: True test: type: 's3dis' data_root: '/workplace/dataset/s3dis/preprocess' prefix: 'Area_5' suffix: '_inst_nostuff.pth' training: False voxel_cfg: scale: 50 spatial_shape: [128, 512] max_npoint: 250000 min_npoint: 5000 x4_split: True

dataloader: train: batch_size: 4 num_workers: 4 test: batch_size: 1 num_workers: 1

optimizer: type: 'Adam' lr: 0.004

fp16: False epochs: 20 step_epoch: 0 save_freq: 2 pretrain: '/workplace/scripts/finetune/2023-02-12-142656-669/work_dirs/softgroup_s3dis_backbone_fold5/latest.pth' #'./checkpoint/softgroup_s3dis_spconv2.pth' work_dir: ''

2023-02-16 21:40:51,876 - INFO - Distributed: True 2023-02-16 21:40:51,876 - INFO - Mix precision training: False 2023-02-16 21:40:56,706 - INFO - Load train dataset: 4080 scans 2023-02-16 21:40:56,707 - INFO - Load test dataset: 68 scans 2023-02-16 21:40:56,708 - INFO - Resume from /workplace/scripts/finetune/2023-02-17-181756-118/work_dirs/softgroup_s3dis_fold5/latest.pth 2023-02-16 21:40:56,887 - INFO - Training 2023-02-16 21:41:10,925 - INFO - Reducer buckets have been rebuilt in this iteration. 2023-02-16 21:41:10,935 - INFO - Reducer buckets have been rebuilt in this iteration. 2023-02-16 21:41:10,935 - INFO - Reducer buckets have been rebuilt in this iteration. 2023-02-16 21:41:10,939 - INFO - Reducer buckets have been rebuilt in this iteration.

Christinepan881 commented 1 year ago

x4_split should be the same in the two training stages.