search

zhihou7 / HOI-CL

Series of work (ECCV2020, CVPR2021, CVPR2021, ECCV2022) about Compositional Learning for Human-Object Interaction Exploration
https://sites.google.com/view/hoi-cl
MIT License
76 stars 11 forks source link

View images by category #6

Closed anjugopinath closed 2 years ago

anjugopinath commented 3 years ago

I downloaded the HICO-DET dataset. Is it possible to view the images by category. For ex) all images with bottles bottle - drink with bottle - hold bottle - carry

etc ?

zhihou7 commented 3 years ago

I think yes.

we have illustrated the feature representation via t-SNE in Figure 10 and Figure 11 which include the features of each object under the same verb category (the same edges in each cluster in Figure 10 (b) and Figure 11(c)). However, we do not illustrate the same object (bottle) with different verbs. But we think it is also possible.

It might be easy to achieve this according to "extract_HO_feature.py", feats_tsne.py, extract_HO_feature_fake.py, feats_tsne_fake.py. feats_tsne_fake.py is the code for illustrating object features. I guess it is easy to change to illustrate verb features according to object category.

Regards

zhihou7 commented 3 years ago

emm, I guess misunderstand your points. Do you mean select out the images according to the object category and verb category?

You can select it according to the annotations. https://drive.google.com/file/d/1M4j5-rHcdfHYVfHQToccO0SsEGP4nGC1/view?usp=sharing. This include verb id and object id for each annotation (box pair).

anjugopinath commented 2 years ago

Thank You so much for your reply.

This is one of the entries: {"image_id": 408, "category_id": 1, "bbox": [216.0, 46.0, 350.0, 354.0], "iscrowd": 0, "id": 1157}

Could you tell me what "category_id" and "id" stands for please? Similarly, what does "iscrowd" stand for?

Does this contain entries for both HICO-DET and HOI-COCO dataset? Do you have a script to download specific images? My end goal is to download all images containing a certain object - for example, download all images associated with object name 'cup' , 'donut' etc.

Thank You in advance!

anjugopinath commented 2 years ago

Also, is it possible to download training images by object category?

zhihou7 commented 2 years ago

Sorry for confusing you. For HICO-DET, the provided file is wrong. Here (https://drive.google.com/drive/folders/1vIDhLwpVSTQ1aPNQh_Qz4xXU8D98TYRD) provides the annotation for verb category and object category.

For example, in this item,

{"file_name": "HICO_train2015_00000001.jpg", "img_id": 1, "annotations": [{"bbox": [207, 32, 426, 299], "category_id": 1}, {"bbox": [58, 97, 571, 404], "category_id": 4}], "hoi_annotation": [{"subject_id": 0, "object_id": 1, "category_id": 73, "hoi_category_id": 153}, {"subject_id": 0, "object_id": 1, "category_id": 77, "hoi_category_id": 154}, {"subject_id": 0, "object_id": 1, "category_id": 88, "hoi_category_id": 155}]}

the items in annotations represent the object category. The items in hoi_annotation mean the hoi category (subject (that's fixed to person), object, category_id( verb id)). In hoi category subject_id and object_id is the index of corresponding object in hoi_annotation.

For the images of HICO-DET, you can download from https://drive.google.com/open?id=1QZcJmGVlF9f4h-XLWe9Gkmnmj2z1gSnk.

For V-COCO, you can convert the annotation according to https://github.com/hitachi-rd-cv/qpic#v-coco.

anjugopinath commented 2 years ago

Thank You so much. This answers my questions I believe.