search

chanyn / HKRM

Hybrid Knowledge Routed Module for Large-scale Object Detection (NerIPS2018)
103 stars 19 forks source link

Attribute & Relationship Knowledge GroundTruth #1

Closed Bedrettin-Cetinkaya closed 5 years ago

Bedrettin-Cetinkaya commented 5 years ago

Hello, I can not find any information about collecting ground truth of attribute and relationship knowledge. How do you get these ground-truth?

Thanks in advance.

chanyn commented 5 years ago

Hi, we described that how to build ground-truth graph in Section 4 Knowledge Graph Construction. In Visual Genome datasets, you can get each object attribute annotation and relationship annotation between objects. In brief, we sum each object attribute annotation of entire dataset if the class of object is in the top 3000 frequent categories we had defined, so we get a 3000D frequency statistic matrix. For relationship frequency statistic matrix, we first new a 30003000 zero matrix Qr. Then we check the ground-truth of each relationship annotation and add 1 to the corresponding position of Qr, if both classes are in the range of top 3000 frequent categories. We think the above operation is easy to implement, and we have provided attribute and relationship frequency statistic matrix in README.

Bedrettin-Cetinkaya commented 5 years ago

Thanks for your quick answer. Actually, i want to ask that how do you collect ground truth for pascal voc and ms-coco dataset? I forgot to specify datasets.

Easyfeng222 commented 5 years ago

@Bedrettin-Cetinkaya Excuse me, have you implemented this paper?

Bedrettin-Cetinkaya commented 5 years ago

@Easyfeng222 , No, I didn' t.

chanyn commented 5 years ago

Oh, we collect coco/voc gt graph by matching classes names with VG. I have released corresponding code in lib/datasets/collect_coco_voc_gtgraph.py First, you should download attribute/relationship frequency statistic matrix. Second, you should save your dataset classes names. We provide VG names (VG_3000_name_to_ind.pkl) and COCO names (coco_name.pkl).