xdeng7
commented
7 months ago Benchmarking results in Sec. B.2 (Table 8~11) show models trained on COCONut-S perform worse on COCO-val than those trained on COCO. Is this because COCO and COCONut have different quality of annotations, resulting in models trained by COCONut-S actually getting better results, but being incorrectly evaluated by COCO-val?
yes, basically better quality data will train better model, since the evaluation set is pretty wrong and thus result in worse performance, because the predictions will do better job than the GT in val set, you can compare the model prediction to the original COCO val set by visualizing some example images.
Benchmarking results in Sec. B.2 (Table 8~11) show models trained on COCONut-S perform worse on COCO-val than those trained on COCO. Is this because COCO and COCONut have different quality of annotations, resulting in models trained by COCONut-S actually getting better results, but being incorrectly evaluated by COCO-val?