HOI-COCO data format

[daoyuan98]

Hi, Thank you again for your great work!

In the Trainval_GT_VCOCO_obj_21.pkl, which contains the split of HOI-COCO, for each training sample, the format seems to be [image_id, action_list1, human_box, object_box, action_list2, object_list]. May I ask what is the difference between action_list1 and action_list2?

[zhihou7]

The second row is the verb label and the 4th row is all verb labels of person ( might include interactions with other objects). the last row is object label. Then you can get the compose label from 2nd row and last row.

[daoyuan98]

Thanks!

[daoyuan98]

Hi, I just found in the provided .pkl file, all of possible class labels are in range (0, 21). I wonder if I downloaded the wrong file or some post-processing are needed?

I downloaded the data from the following links. https://cloudstor.aarnet.edu.au/plus/s/YEiPiX0B3jaFasU https://cloudstor.aarnet.edu.au/plus/s/6NzReMWHblQVpht

[zhihou7]

@daoyuan98, sorry for getting confusing you. I might provide wrong information to you in the last message. The second row is the verb label and the 4th row is all verb labels of person ( might include interactions with other objects). the last row is object label.

You can get the composition label as this line:

https://github.com/zhihou7/HOI-CL/blob/735c6940b75c504399ece9e801eeb25924f2da21/lib/ult/ult.py#L642-L676

Sincerely sorry for the last comments.

Regards,

[daoyuan98]

Noted! Thank you so much for your reply!

[daoyuan98]

Hi, Sorry to open this issue again. I still have some question about the HOI-COCO dataset. I just wanted to check, (1) Is the results reported in ATL on HOI-COCO considers 21 verb classes and the train file is Trainval_GT_VCOCO_obj_21.pkl? (2) Which test set did you use? I found only 4969 images in that file, plus vcoco official test set, their is only 9915 images, which is different from 10346 reported in paper.

Sincerely hope you can help!

[zhihou7]

@daoyuan98, Sorry for getting you latter, 1), Yes, it is Trainval_GT_VCOCO_obj_21.pkl. 2), The test set is from this file: "Test_Faster_RCNN_R-50-PFN_2x_HICO_DET.pkl" in https://cloudstor.aarnet.edu.au/plus/s/9YxboaI6d5lITLk.

Sorry for getting confusing you. The numbers of images reported in paper is directly copied from iCAN. And those files are also followed from the released pkls in iCAN. I did not check the images, and I think it is better to refer to the two files: Trainval_GT_VCOCO_obj_21.pkl and Test_Faster_RCNN_R-50-PFN_2x_HICO_DET.pkl.

[daoyuan98]

Thank you so much for your reply! I am sorry to have to bother you again.. It seems Test_Faster_RCNN_R-50-PFN_2x_HICO_DET.pkl has only detection annotation (not hoi detection results). Besides, it has 9658 images (with id from 1 - 9767), but I didn't find how these id correlates with that of VCOCO.. Did I miss any files?

Thanks again for your reply!

[zhihou7]

"lib/ult/vsrl_eval2.py" includes the evaluation process.

def get_vcoco2():
    from ult.vsrl_eval2 import VCOCOeval
    vcocoeval = VCOCOeval(cfg.DATA_DIR + '/' + 'v-coco/data/vcoco/vcoco_test.json',
                      cfg.DATA_DIR + '/' + 'v-coco/data/instances_vcoco_all_2014.json',
                      cfg.DATA_DIR + '/' + 'v-coco/data/splits/vcoco_test.ids')
    return vcocoeval

Those files are generated after you have installed v-coco dataset.

[daoyuan98]

Thanks! So from my understanding, HOI-COCO actually takes the same test set with VCOCO, but with different number of actions (21), and split (rare, non-rare)?

[zhihou7]

Yes!

[daoyuan98]

Thank you so much!