I run the shell script directly but cannot got the result on the paper

[herbertchen1]

Only unseen=8.8357, seen=87.9500, h=16.0581, unseen class accuracy= 64.2205472255

May I ask which hyper parameter would influence the result so much?

[ahangchen]

I'm not the author but I think it's related to this issue #1

[akku1506]

@herbertchen1 Can I please know what you are trying to do?

[herbertchen1]

i hope to use your method into a NLP problem, but i found it's hard to got a desirable result in my task， so i currently give up the task. and thanks for your reply.

[akku1506]

Okay!

[herbertchen1]

but i'm curious if the result i got with the script is right?

[akku1506]

@herbertchen1 You ran the results for AWA2 and you got the results described above! The unseen class accuracy in the conventional zero-shot setting approaches to that of AWA i.e. 66.4%. There could be this much difference in the reported results from the paper as AWA is similar to AWA2 but not exactly the same. While for generalized zero-shot setting, I think you ran the model for more number of epochs than 30 which could be a reason for this performance drop. I just suspect it, please let me know if I am correct or you did something else!

[106084]

I also can't get the results I want in the following configuration.

Namespace(attSize=85, batch_size=64, beta1=0.5, class_embedding='att', classifier_checkpoint=49, classifier_lr=0.001, classifier_modeldir='./models_classifier', cls_weight=0.01, critic_iter=5, dataroot='/home/suxiangdong/inpainting/lisgan/xlsa17/data', dataset='AWA1', gzsl=True, image_embedding='res101', lambda1=10.0, logdir='logs_awa', lr=1e-05, manualSeed=9182, matdataset=True, modeldir='models_awa', nclass_all=200, ndh=4096, nepoch=30, netD_name='MLP_CRITIC', netG_name='MLP_G', ngh=4096, nz=85, preprocessing=True, print_every=1, resSize=2048, save_every=100, standardization=False, start_epoch=0, syn_num=300, val_every=1, validation=False)

Model loaded unseen=9.8991, seen=84.2037, h=17.7155

and in CUB Namespace(attSize=312, batch_size=64, beta1=0.5, class_embedding='att', classifier_checkpoint=49, classifier_lr=0.001, classifier_modeldir='./models_classifier', cls_weight=0.01, critic_iter=5, dataroot='/home/wwy/zsl/data', dataset='CUB', gzsl=False, image_embedding='res101', lambda1=10.0, logdir='logs_awa', lr=1e-05, manualSeed=3483, matdataset=True, modeldir='models_awa', nclass_all=200, ndh=4096, nepoch=70, netD_name='MLP_CRITIC', netG_name='MLP_G', ngh=4096, nz=312, preprocessing=True, print_every=1, resSize=2048, save_every=100, standardization=False, start_epoch=0, syn_num=300, val_every=1, validation=False)

unseen class accuracy= 45.440014845014794

[flywithcloud]

Excuse me. I also can't reproduce the results in the original paper (Feature Generating Networks for Zero-Shot Learning) via this code. Specifically, even by using the model provided by @akku1506, I can't reproduce the GZSL performance on AWA2 datasets, which is {seen:90.8%, unseen:7.5%, H:13.9%} in my experiment, a big gap to the results of AWA1 in the original paper. though the achieved ZSL performance (unseen: 66.9%) of this model is close to the results of AWA1 in the original paper. However, what confuses me is not the low performance of GZSL, which may be related to the hyper parameter. what really confused me is the reason why the same generated visual feature (e.g. feature generated by model provided by akku1506) can achieve reasonable performance on ZSL task while being not able to achieve good performance on GZSL task. Specifically speaking, dose this method provided here really solve the problem about prediction bias to seen classes. Anyway, many thanks to akku1506 for her kind sharing