does anyone can reproduce the paper result?

[AndyYuan96]

Hi, everyone, can you reproduce the paper result on val dataset use this repo? I can't reproduce. I first follow Det3d's tutorial to install Det3d, and prepare the info data and reduced data, and then use se-ssd to prepare gtaug data. To run the code, I change some code following others in other question, just change model_ema = build_detector(), other change is the path problem and some hard code batch_size, not the core code. For pertained model, I use the model provided in CIA-SSD repo. What's more, I use 2 v100 32g gpu, and batch_size = 1 for each gpu, As I can only run the code with batch_size = 1, I don't know why now, so the total batch_size is 2 with 2 v100 32g gpu. For the result, after SE-SSD training, compared with pertained model, the evaluation result is generally the same, don't have the improvement in paper. So I just wonder does anyone can reproduce the result? Can you point out where I'm wrong, thank you very much. @Vegeta2020 ,@pigtigger, @WWW2323.

[AndyYuan96]

For Det3d install, I just follow the tutorial, use the poodarchu's spconv, and then install Det3d.

[WWW2323]

No, i gave up. If you make it, please @ me, thanks.

[maudzung]

Lol. Me too 😂

[yujmo]

Evaluation official: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:88.58, 77.79, 75.09 bev AP:84.71, 75.34, 67.57 3d AP:47.61, 41.61, 40.11 aos AP:87.11, 75.19, 72.10 car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:88.58, 77.79, 75.09 bev AP:90.81, 89.37, 87.25 3d AP:90.62, 88.13, 79.81 aos AP:87.11, 75.19, 72.10

Evaluation coco: car coco AP@0.50:0.05:0.95: bbox AP:60.11, 56.47, 54.31 bev AP:53.67, 50.72, 47.66 3d AP:41.55, 37.92, 35.54 aos AP:59.18, 54.57, 52.16

[Y-why]

My trained best results is slightly lower than the released one...the training is a little tricky, but I finally make it :)

[WWW2323]

@Y-why Great!! Could you share your training procedure?

[Y-why]

@WWW2323 My advice is to use the ODIoU loss to train CIA-SSD, thus get a better pre-trained model, which can make it easier to get a decent performance.

[WWW2323]

@Y-why thanks!!

[Eaphan]

My trained best results is slightly lower than the released one...the training is a little tricky, but I finally make it :)

@Y-why Is the left image result of training CIA-SSD with ODIoU loss? It seems nice on moderate but terrible on simple and hard cases.

[Eaphan]

@Y-why And what's the performance using the ODIoU loss to train CIA-SSD? The performance doesn't improve obviously with the ODIoU loss.

[Y-why]

@Viczyf The left and right pictures show the AP_11 and AP_40 of SE-SSD, respectively. The ODIoU loss can improve the CIA-SSD a bit and help to reproduce the SE-SSD's results. For the improvement, you may refer to the issue.

[Eaphan]

@WWW2323 Have your ever reproduced the performance successfully?

[cytehsi]

Evaluation official_AP_11: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:98.58, 90.18, 89.67 bev AP:90.57, 88.85, 88.17 3d AP:90.19, 85.81, 79.17 aos AP:98.52, 89.93, 89.26

car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:98.58, 90.18, 89.67 bev AP:98.66, 90.29, 89.87 3d AP:98.61, 90.26, 89.81 aos AP:98.52, 89.93, 89.26

Evaluation official_AP_40: car AP(Average Precision)@0.70, 0.70, 0.70: bbox AP:99.53, 95.61, 93.19 bev AP:96.71, 92.04, 89.63 3d AP:93.76, 85.99, 83.37 aos AP:99.48, 95.30, 92.71

car AP(Average Precision)@0.70, 0.50, 0.50: bbox AP:99.53, 95.61, 93.19 bev AP:99.57, 95.94, 95.57 3d AP:99.56, 95.87, 95.44 aos AP:99.48, 95.30, 92.71