使用您提供的训练模型进行可视化测试，我的推理结果有些小问题

Mikeworld-git commented 2 years ago

❓ Questions and Help

感谢您提供的代码。但是我在测试您的可视化代码时，发现推理结果有些问题。我是用的是您训练好的模型upload_causal_motif_sgdet。在数据集VG随机选取几张进行测试。我发现都会出现类似问题。

这是我测试时候运行的代码。 python -m torch.distributed.launch --master_port 10027 --nproc_per_node=1 tools/relation_test_net.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX False MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR CausalAnalysisPredictor MODEL.ROI_RELATION_HEAD.CAUSAL.EFFECT_TYPE TDE MODEL.ROI_RELATION_HEAD.CAUSAL.FUSION_TYPE sum MODEL.ROI_RELATION_HEAD.CAUSAL.CONTEXT_LAYER motifs TEST.IMS_PER_BATCH 1 DTYPE "float16" GLOVE_DIR /content/drive/MyDrive/checkpoint/glove MODEL.PRETRAINED_DETECTOR_CKPT /content/drive/MyDrive/checkpoint/upload_causal_motif_sgdet OUTPUT_DIR /content/drive/MyDrive/checkpoint/upload_causal_motif_sgdet TEST.CUSTUM_EVAL True TEST.CUSTUM_PATH /content/drive/MyDrive/checkpoint/testimage DETECTED_SGG_DIR /content/drive/MyDrive/checkpoint/output

部分可视化代码输出结果： box_labels 0: shirt; score: 0.8227353692054749 box_labels 1: jean; score: 0.7995691895484924 box_labels 2: man; score: 0.793093740940094 box_labels 3: jean; score: 0.6989970207214355 box_labels 4: shirt; score: 0.6820546388626099 box_labels 5: woman; score: 0.6755836009979248

rel_labels 0: 5_woman => wearing => 1_jean; score: 0.735344648361206 rel_labels 1: 2_man => wearing => 3_jean; score: 0.7006661295890808 rel_labels 2: 5_woman => wearing => 0_shirt; score: 0.6986700892448425 rel_labels 3: 2_man => wearing => 1_jean; score: 0.6861620545387268 rel_labels 4: 2_man => wearing => 4_shirt; score: 0.6859005093574524 rel_labels 5: 5_woman => wearing => 3_jean; score: 0.6535178422927856

我发现2_man => wearing => 3_jean的score比2_man => wearing => 1_jean的score更高。请问是我哪里参数设置的有问题吗？

KaihuaTang commented 2 years ago

Due to the existence of the background category (NO_RELATIONSHIP), applying TDE may cause a ranking problem. This is why TDE has high mean Recall but low Recall.

To solve this problem, you need to use the Background-Exempted Inference trick proposed in another paper, the details of the Background-Exempted Inference trick is introduced in https://github.com/KaihuaTang/Long-Tailed-Recognition.pytorch/tree/master/lvis1.0

Mikeworld-git commented 2 years ago

感谢您的回答，我去学习相关内容。谢谢您