search

skyhehe123 / SA-SSD

SA-SSD: Structure Aware Single-stage 3D Object Detection from Point Cloud (CVPR 2020)
492 stars 106 forks source link

Result reproduction. #31

Closed outsidercsy closed 4 years ago

outsidercsy commented 4 years ago

Hi, @skyhehe123. Thanks for your awesome work. I meet problems when reproducing the result. Following is my result. I keep the same config as in the car_cfg.py. But it seems that an obvieous gap exists. image I notice that in the commit history, the training shedule has changed a lot (from cosine sgd to adam). Is that the problem? Could you please offer me some suggestions?

outsidercsy commented 4 years ago

I tried the code of the earliest commit(e8d4db), but the result was still unsatisfactory. image

skyhehe123 commented 4 years ago

@outsidercsy. Hi, if you are using pytorch 1.4, try to add align_corners=True at https://github.com/skyhehe123/SA-SSD/blob/9bb2ef4aecc7206ea935977d45781267b8a15001/mmdet/models/single_stage_heads/ssd_rotate_head.py#L372

skyhehe123 commented 4 years ago

@outsidercsy This results seems to be the bug and error instead of the reproduction of the repo

outsidercsy commented 4 years ago

@skyhehe123. I have tried to rebuild the conda env, but the result is the same. The env detail is listed below. Could you please tell me what is the difference with yours? I download the pretrained model and do evaluation. The result is the same as what you have posted. I suppose the bug may lie in training phase. By the way, is the create_data.py in your project the same as that in second.pytorch? I use the create_data.py in second.pytorch but not yours to prepare kitti data. image

outsidercsy commented 4 years ago

I find the reason. I only use the cars in the train set for data augmentation. But the result posted in this repo is based on the augmentation of cars of trainval set. I think it is not proper.

skyhehe123 commented 4 years ago

@outsidercsy The augmentation is also based on the train set. https://github.com/skyhehe123/SA-SSD/blob/9bb2ef4aecc7206ea935977d45781267b8a15001/configs/car_cfg.py#L103

oby-yang commented 4 years ago

create_data.py doesn't generated 'kitti_dbinfos_train.pkl', it only generated 'kitti_infos_train.pkl' and 'kitti_dbinfos_trainval.pkl. so which should i use?

Divadi commented 4 years ago

If you're testing on val split, you should definitely not use kitti_db_infos_trainval. At the end of create_data.py, you should add:

create_groundtruth_database(data_path=YOUR_DATA_PATH, \
                                info_path=None, \  
                                db_info_save_path=None)

that will create kitti_dbinfos_train.pkl for you from kitti_infos_train.pkl.

if you're testing on the test split, maybe use kitti_dbinfos_trainval.pkl for more augmentation?

vehxianfish commented 3 years ago

@outsidercsy .Do you reproduce the result.Can your share your steps here?Or the lib version effect on the result,eg.spconv? thank you!

vehxianfish commented 3 years ago

@Divadi .Hi,Do you reproduce result On 3D mid with 84.40? I only has 84.05 with 3.5 difference. Thank you very much. Do you data augmentation includes Groudtruth augmentation?