About the part localization accuracy

[lntzm]

Hello dear author, The experiments about part localization in Section 4.2 are interesting, and I wanna run this part in CUB dataset. I have noticed that there is a function test_with_IoU in model/main_utils.py and thus I simply change:

acc_ZSL = test_zsl(opt, model, testloader_unseen, attribute_zsl, data.unseenclasses)

(model/main.py, line 128) to:

train_body_avg_IoU, train_mean_IoU = test_with_IoU(opt, model, trainloader, attribute_zsl, data.seenclasses,
                                       save_att='./vis_out', sub_group_dic=sub_group_dic, group_dic=group_dic)
seen_body_avg_IoU, seen_mean_IoU = test_with_IoU(opt, model, testloader_seen, attribute_zsl, data.seenclasses,
                                       save_att='./vis_out', sub_group_dic=sub_group_dic, group_dic=group_dic)
unseen_body_avg_IoU, unseen_mean_IoU = test_with_IoU(opt, model, testloader_unseen, attribute_zsl, data.unseenclasses,
                                       save_att='./vis_out', sub_group_dic=sub_group_dic, group_dic=group_dic)
print(f'train_body_avg_IoU:\n {train_body_avg_IoU}')
print(f'train_mean_IoU:\n {train_mean_IoU}')
print(f'seen_body_avg_IoU:\n {seen_body_avg_IoU}')
print(f'seen_mean_IoU:\n {seen_mean_IoU}')
print(f'unseen_body_avg_IoU:\n {unseen_body_avg_IoU}')
print(f'unseen_mean_IoU:\n {unseen_mean_IoU}')

And the model I used is the CUB_ZSL_id_0.pth you provided in readme, and then run script/CUB_ZSL_eval.sh. However, I got a bad result as follows, which is much lower than in the paper.

train_body_avg_IoU:
 {'head': [0.15879228403690243], 'breast': [0.16974481883547157], 'belly': [0.08828276307295028], 'back': [0.018849568284081235], 'wing': [0.07151062607696726], 'tail': [0.0011827321111768185], 'leg': [0.09589752466678207]}
train_mean_IoU:
 10.051293082885914
seen_body_avg_IoU:
 {'head': [0.1503850541928123], 'breast': [0.1777520724889146], 'belly': [0.08024691358024691], 'back': [0.021739130434782608], 'wing': [0.07607843137254902], 'tail': [0.0011811023622047244], 'leg': [0.10732054015636106]}
seen_mean_IoU:
 10.22536903709444
unseen_body_avg_IoU:
 {'head': [0.16280549898167007], 'breast': [0.17755102040816326], 'belly': [0.08723909511311086], 'back': [0.02199736031676199], 'wing': [0.08664343786295005], 'tail': [0.0002257336343115124], 'leg': [0.06950582563278425]}
unseen_mean_IoU:
 10.095703971924008

And here is the result of CUB_GZSL_id_0.pth:

train_body_avg_IoU:
 {'head': [0.15472202423378476], 'breast': [0.17549146218969722], 'belly': [0.11056831134994301], 'back': [0.023660275319567353], 'wing': [0.07833698030634573], 'tail': [0.002328115712304339], 'leg': [0.1538327221931203]}
train_mean_IoU:
 11.610196259874305
seen_body_avg_IoU:
 {'head': [0.15651739874500856], 'breast': [0.1744746481588587], 'belly': [0.10163139329805997], 'back': [0.02273863068465767], 'wing': [0.08019607843137254], 'tail': [0.0033464566929133858], 'leg': [0.1613361762615494]}
seen_mean_IoU:
 11.614905426325112
unseen_body_avg_IoU:
 {'head': [0.1674304141208418], 'breast': [0.1717687074829932], 'belly': [0.11023622047244094], 'back': [0.026836779586449623], 'wing': [0.09349593495934959], 'tail': [0.0019187358916478556], 'leg': [0.13097629570108477]}
unseen_mean_IoU:
 11.679072538719334

So:

• Is this operation right?
• Could you provide more detailed information about this part?

Looking forward to your reply & thanks a lot !