search

thunlp / VisualDS

MIT License
25 stars 3 forks source link

What CCKB and VGKB means? #4

Closed guikunchen closed 2 years ago

guikunchen commented 2 years ago

In training scrpit, DATASETS.TRAIN is set ts 20DSVG CC / VG KB_train. However, experimental setting says "during training distant supervision is performed using the intersection of relations from Visual Genome and the knowledge base", which means that there is 70 relation categories during training.

The second question is what CC means?

waxnkw commented 2 years ago

Thanks for your interest.

20vs50: In 20DSVG CC / VG KB_train, 20 means the total number of relation categories in experiments. We have two settings including 20 categories and 50 categories. The result reported in the main paper is 20 categories setting, and 50 categories are reported in the appendix.

VG vs CC: CCKB is KB constructed from Conceptual Caption a web image-text pairs dataset. When we assume that we have no visual relation annotations (weakly supervised), CCKB can be used. However, when we have the full annotated VG (semi-supervised), we can just construct an in-domain KB from VG's relation annotations, which is VGKB.

CCKB for training: original CCKB has over 1,000 relation categories. However, many categories do not exist in 20 settings or 50 settings. To make an evaluation, we align the CCKB with the experiments' setting. For example, in 20 settings, only the pre-defined 20 relation categories are retained, leaving others ignored.

guikunchen commented 2 years ago

@waxnkw Thanks for your detailed explanation!