search

shinya7y / UniverseNet

USB: Universal-Scale Object Detection Benchmark (BMVC 2022)
Apache License 2.0
422 stars 54 forks source link

Very low AP when using cfg.data.val to build the dataset #10

Closed Jossome closed 3 years ago

Jossome commented 3 years ago

Hi, I'm currently trying universenet on Waymo dataset. I used your WaymoCOCO code to generate f0 training and validation set.

After training, I run tools/test.py with the default setting. It loads 4038 samples for validation, and the performance is good. However, when I change this line to dataset = build_dataset(cfg.data.val), the progress bar shows only 3945 samples, and the resulting AP is very very low (close to 0).

I checked the config file - the cfg.data.val has exactly the same field as cfg.data.test.

What was going wrong in this case?

shinya7y commented 3 years ago

Is test_mode set to True by replacing other cfg.data.test in test.py?

shinya7y commented 3 years ago

For changing test data, it's better to edit configs or use --cfg-options. Example for COCO test: --cfg-options "data.test.ann_file=data/coco/annotations/image_info_test-dev2017.json" "data.test.img_prefix=data/coco/test2017/"

Jossome commented 3 years ago

Is test_mode set to True by replacing other cfg.data.test in test.py?

Yes this solves my problem. When I change it to cfg.data.val, I also need to set cfg.data.val.test_mode=True, as in here in test.py.

Thanks!

Jossome commented 3 years ago

@shinya7y Hi, I have a small question about training on waymo, so i'm just putting it in this thread - For the models on waymo datasets provided here, did you train them with images of all 5 cameras or just the front camera?

shinya7y commented 3 years ago

I trained them with images of all 5 cameras.