May I kindly ask why each sample in outputs_class_rel = self.rel_class_embed(torch.cat((hs_sub, hs_obj, so_masks), dim=-1)) at models/reltr.py:111 has dim 52, instead of 51? The processed VG has 50 relation classes, so I assume the dim should be 51 with an additional no relation ('background') class.
Besides, it can be seen from models/reltr.py:213-214
Count the number of predictions that are NOT "no-object" (which is the last class)
card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1)
that the last class represents the "no-object" class.
I notice that REL_CLASSES has a length of 51, not 52, and 'background' is at the index 0, not at the last index.
Is this REL_CLASSES in colab the label indices you use in your training code (in data/vg/rel.json)? Because I am working on re-organizing the dataset labels for my own project, I need to know the exact ordering of these label indices. Thanks for your assistance!
May I kindly ask why each sample in outputs_class_rel = self.rel_class_embed(torch.cat((hs_sub, hs_obj, so_masks), dim=-1)) at models/reltr.py:111 has dim 52, instead of 51? The processed VG has 50 relation classes, so I assume the dim should be 51 with an additional no relation ('background') class.
Besides, it can be seen from models/reltr.py:213-214 Count the number of predictions that are NOT "no-object" (which is the last class) card_pred = (pred_logits.argmax(-1) != pred_logits.shape[-1] - 1).sum(1) that the last class represents the "no-object" class.
However, in the colab notebook, you said REL_CLASSES = ['background', 'above', 'across', 'against', 'along', 'and', 'at', 'attached to', 'behind', 'belonging to', 'between', 'carrying', 'covered in', 'covering', 'eating', 'flying in', 'for', 'from', 'growing on', 'hanging from', 'has', 'holding', 'in', 'in front of', 'laying on', 'looking at', 'lying on', 'made of', 'mounted on', 'near', 'of', 'on', 'on back of', 'over', 'painted on', 'parked on', 'part of', 'playing', 'riding', 'says', 'sitting on', 'standing on', 'to', 'under', 'using', 'walking in', 'walking on', 'watching', 'wearing', 'wears', 'with']
I notice that REL_CLASSES has a length of 51, not 52, and 'background' is at the index 0, not at the last index. Is this REL_CLASSES in colab the label indices you use in your training code (in data/vg/rel.json)? Because I am working on re-organizing the dataset labels for my own project, I need to know the exact ordering of these label indices. Thanks for your assistance!