Jingkang50 / OpenPSG

Benchmarking Panoptic Scene Graph Generation (PSG), ECCV'22
https://psgdataset.org
MIT License
409 stars 68 forks source link

[Feature] Support PSG Challenge Test Code #5

Closed Jingkang50 closed 2 years ago

Jingkang50 commented 2 years ago

To support PSG challenge on codalab, we need to provide the following things:

OpenPSG codebase:

Reference Data:

To put ground-truth data on codalab, we should provide a data_val directory and a psg_val.json with validation data only. We should then fix the config option in the tools/grade.py with a new config file of configs/_base_/datasets/psg_val.py.

The deadline of this feature is the following Monday, 5pm. Then @Jingkang50 will set up the competition.

We shall track our ideas and designs in this issue.

Jingkang50 commented 2 years ago

We should strictly follow the file structure in the submission.zip as below.

submission.zip
    ├── relation.json
    └─── panseg
        ├── 0.png
        ├── 1.png
        └── ...

Make sure that you need to choose all the files and then compress them, and be sure there is no subdirectory in the folder. We would not have a submission directory. Let's directly zip relation.json and panseg, as the convention of codalab.

relation.json should look like follows.

    [{"relations": [[3, 5, 1], [3, 2, 1], [0, 2, 4], [3, 1, 1], [1, 2, 4], 
                    [5, 2, 4], [3, 4, 1], [3, 7, 1], [3, 0, 1], [4, 2, 4], 
                    [7, 2, 4], [2, 4, 6], [2, 5, 3], [0, 4, 4], [5, 1, 3], 
                    [5, 4, 3], [5, 7, 3], [5, 0, 3], [5, 3, 3], [2, 0, 3],],
      "segments_info": [{"category_id": "14", "id": 3205072}, 
                        {"category_id": "14", "id": 705906}, 
                        {"category_id": "126", "id": 15037987}, 
                        {"category_id": "120", "id": 2834972}, 
                        {"category_id": "118", "id": 1268710}, 
                        {"category_id": "117", "id": 12333373}, 
                        {"category_id": "102", "id": 8492910}, 
                        {"category_id": "92", "id": 11194259}], 
      "pan_seg_file_name": "0.png"}, ...]

We should provide a full example of submission.zip with 2177 images.