How to test the test2017-dataset and get the test_dev_results?

[reedwiththought]

Hi, thanks for your great work. I try to test the test2017-dataset and want to get the test_dev_results on myself, but failed. Could you give me some advice? Thank you.

Do I need to get this file---'instances_test_dev2017.json'? If need, where can I find it?

[chensnathan]

You should upload your test_dev results to the panoptic evaluation server to get the performance on the test_dev split.

[reedwiththought]

Thanks for your reply. I try to get my test_dev results (panoptic_test_pred.json) but failed. Could you tell me how to solve this problem? Thank you.

After I get my test_dev results (panoptic_test_pred.json and segmentations_folder_test_pred), I will upload it to the panoptic evaluation server.

I change img_prefix path = test2017 in coco_panoptic.py, but I don't have _panoptic_test2017_detection_format_thingsonly.json.

val=dict(
    type=dataset_type,
    ann_file=data_root + 'annotations/panoptic_val2017_detection_format_things_only.json',
    img_prefix=data_root + 'test2017/',
    with_panoptic=True,
    things_other=True,
    pipeline=test_pipeline),
test=dict(
    type=dataset_type,
    ann_file=data_root + 'annotations/panoptic_val2017_detection_format_things_only.json',
    img_prefix=data_root + 'test2017/',
    with_panoptic=True,
    things_other=True,
    pipeline=test_pipeline))

[chensnathan]

For the test split, we just need the image infos, thus it is ok to use instances_test_dev2017.json. See here for more detailed settings.

[chensnathan]

You can download the instances_test_dev2017.json on the official website of MS COCO.

[reedwiththought]

OK, thank you very much.