open-mmlab / mmdetection

OpenMMLab Detection Toolbox and Benchmark
https://mmdetection.readthedocs.io
Apache License 2.0
29.71k stars 9.48k forks source link

FCOS + Cityscapes -> insufficient accuracy #8364

Open BayMaxBHL opened 2 years ago

BayMaxBHL commented 2 years ago

Question:

I use the Cityscapes dataset for testing target detection. Using the same training strategy (1 gpu) and dataset, the models are FasterRCNN, FCOS. The FasterRCNN replication results are consistent with the official map, which is 42. However, when using FCOS training, you can only get about 30. The results of the two training models on COCO are approximately the same, but their performance on Cityscapes is quite different.

Seek help:

For Cityscapes datasets, can the authorities provide training results for more models? Because COCO is relatively large, Cityscapes can provide faster reference in the early stages.

BayMaxBHL commented 2 years ago

FasterRCNN+Cityscapes: Faster-RCNN-2048

BayMaxBHL commented 2 years ago

FCOS + Cityscapes image

BayMaxBHL commented 2 years ago

Fcos model-cfg is fcos_r50_caffe_fpn_gn-head_1x_coco.py . Number of categories changed to 8.

BayMaxBHL commented 2 years ago

Later, I tried to change the batch size, the learning rate configuration, and the stripe of fcos head, but there was no improvement.

BayMaxBHL commented 2 years ago

In addition, not using the pre-training (coco) will produce worse results. image

BayMaxBHL commented 2 years ago

Later, I tried to use cloud services (2 GPUs), use the official configuration of fasterrcnn of cityscapes, and only replace the model part for training. ATSS quickly reached 36+, while FCOS finally reached less than 30. image

BayMaxBHL commented 2 years ago

I sincerely hope that mmdetection can provide a baseline for target detection of cityscapes without using the pre training model. Publicize the results of multiple models.

ZwwWayne commented 2 years ago

Some parameters of FCOS and ATSS might need to be specifically tuned for Cityscapes. It is common to fine-tune the model for cityscapes using pre-trained models from COCO.