a cross-modal label mismatch issue

[wangsen-zy]

Hello, while using your code for training, I found that when the calculation reached this point:

Rgb2ir Labels=torch. sensor ([r2i [key. item()] for key in labels Rgb]). cuda()

I have encountered a cross-modal label mismatch issue, for example, I received a label with number 857 identity in labels_ Rgb, but the corresponding cannot be found in r2i.

May I ask if you have encountered such a bug and how did you solve it? Thank you very much for your assistance and answer. The complete error report is attached as follows: ........,.............................................................................................. Epoch: [1][10/200] Time 0.110 (0.114) Data 0.000 (0.003) Loss 14.617 (19.576) Loss ir 8.726 Loss rgb 4.431 Loss cross 5.837
Epoch: [1][20/200] Time 0.107 (0.111) Data 0.000 (0.002) Loss 10.039 (15.427) Loss ir 6.030 Loss rgb 2.347 Loss cross 6.648
Epoch: [1][30/200] Time 0.107 (0.110) Data 0.000 (0.001) Loss 3.919 (13.358) Loss ir 1.985 Loss rgb 1.428 Loss cross 2.024
Epoch: [1][40/200] Time 0.108 (0.109) Data 0.000 (0.001) Loss 5.214 (12.468) Loss ir 2.274 Loss rgb 2.425 Loss cross 2.059
Epoch: [1][50/200] Time 0.108 (0.109) Data 0.000 (0.001) Loss 5.132 (11.942) Loss ir 2.724 Loss rgb 1.715 Loss cross 2.772
Epoch: [1][60/200] Time 0.111 (0.109) Data 0.002 (0.001) Loss 10.027 (11.647) Loss ir 5.210 Loss rgb 3.386 Loss cross 5.723
Epoch: [1][70/200] Time 0.109 (0.109) Data 0.000 (0.001) Loss 5.691 (11.325) Loss ir 2.157 Loss rgb 2.951 Loss cross 2.333
Epoch: [1][80/200] Time 0.110 (0.109) Data 0.000 (0.001) Loss 2.710 (11.161) Loss ir 1.102 Loss rgb 1.259 Loss cross 1.398
Epoch: [1][90/200] Time 0.109 (0.109) Data 0.000 (0.001) Loss 4.867 (10.931) Loss ir 2.357 Loss rgb 1.907 Loss cross 2.415
Epoch: [1][100/200] Time 0.108 (0.109) Data 0.000 (0.001) Loss 4.053 (10.747) Loss ir 1.937 Loss rgb 1.642 Loss cross 1.898
Epoch: [1][110/200] Time 0.109 (0.109) Data 0.000 (0.001) Loss 6.027 (10.657) Loss ir 2.476 Loss rgb 2.926 Loss cross 2.496
Epoch: [1][120/200] Time 0.112 (0.109) Data 0.000 (0.001) Loss 2.776 (10.502) Loss ir 1.463 Loss rgb 0.988 Loss cross 1.299
Epoch: [1][130/200] Time 0.108 (0.109) Data 0.000 (0.001) Loss 4.749 (10.450) Loss ir 1.896 Loss rgb 2.271 Loss cross 2.329
Epoch: [1][140/200] Time 0.110 (0.109) Data 0.000 (0.000) Loss 3.730 (10.353) Loss ir 1.956 Loss rgb 1.283 Loss cross 1.966
Epoch: [1][150/200] Time 0.112 (0.109) Data 0.000 (0.000) Loss 6.019 (10.293) Loss ir 3.195 Loss rgb 1.914 Loss cross 3.641
Epoch: [1][160/200] Time 0.107 (0.109) Data 0.000 (0.000) Loss 3.519 (10.210) Loss ir 2.164 Loss rgb 0.823 Loss cross 2.127
Epoch: [1][170/200] Time 0.108 (0.109) Data 0.000 (0.000) Loss 8.005 (10.191) Loss ir 2.782 Loss rgb 1.775 Loss cross 13.790
Epoch: [1][180/200] Time 0.109 (0.109) Data 0.000 (0.000) Loss 6.111 (10.112) Loss ir 1.851 Loss rgb 3.644 Loss cross 2.463
Epoch: [1][190/200] Time 0.109 (0.109) Data 0.000 (0.000) Loss 2.673 (10.030) Loss ir 1.457 Loss rgb 0.868 Loss cross 1.392
Epoch: [1][200/200] Time 0.109 (0.109) Data 0.000 (0.000) Loss 4.477 (10.021) Loss ir 1.847 Loss rgb 2.186 Loss cross 1.774
==> Create pseudo labels for unlabeled RGB data Extract Features: [50/87] Time 0.483 (0.765) Data 0.000 (0.043)
==> Create pseudo labels for unlabeled IR data Computing jaccard distance... Jaccard distance computing time cost: 53.50986194610596 Computing jaccard distance... Jaccard distance computing time cost: 155.3426811695099 ==> Statistics for IR epoch 2: 386 clusters ==> Statistics for RGB epoch 2: 876 clusters Progressive Graph Matching Progressive Graph Matching Done /opt/anaconda3/envs/SMART/lib/python3.9/site-packages/torchvision/transforms/transforms.py:329: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum. warnings.warn( Traceback (most recent call last): File "/T2023074/uslVI/uslVI/train_sysu.py", line 1110, in main() File "/T2023074/uslVI/uslVI/train_sysu.py", line 378, in main main_worker_stage2(args,log_s1_name,log_s2_name) # Stage 2 File "/T2023074/uslVI/uslVI/train_sysu.py", line 898, in main_worker_stage2 trainer.train(epoch, train_loader_ir,train_loader_rgb, optimizer, print_freq=args.print_freq, train_iters=len(train_loader_ir), i2r=i2r, r2i=r2i) File "/T2023074/uslVI/uslVI/clustercontrast/trainers.py", line 81, in train rgb2ir_labels = torch.tensor([r2i[key.item()] for key in labels_rgb]).cuda() File "/T2023074/uslVI/uslVI/clustercontrast/trainers.py", line 81, in rgb2ir_labels = torch.tensor([r2i[key.item()] for key in labels_rgb]).cuda() KeyError: 857

Process finished with exit code 1

[zesenwu23]

Hi, thanks for your attention. This problem is happened due to the number of RGB clusters is more than twice the number of IR clusters (in your case: 876 > 2 * 386). We implented two rounds of matching in our code. Another same issue can be refered #2 , which we think may help you.

Contact me via email anytime.

[wangsen-zy]

Thank you very much for your reply. This is very helpful to me. I think I know how to modify it