search

yuleiniu / cfvqa

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias
Apache License 2.0
116 stars 14 forks source link

Maybe something wrong in cfvqasimple.py #4

Closed Mike4Ellis closed 3 years ago

Mike4Ellis commented 3 years ago

Hello, thanks for sharing your code! I found a possible wrong in cfvqasimple.py Isn't out['logits_all'] = z_qkv # for optimization should be out['logits_all'] = z_qk # for optimization ? Or I took it wrong?

hannajiang commented 3 years ago

Hello, I want to ask a question. Can you get the result in paper, I got very low result in all files of .yaml; And I didn't modify any parameter.

yuleiniu commented 3 years ago

Hello, thanks for sharing your code! I found a possible wrong in cfvqasimple.py Isn't out['logits_all'] = z_qkv # for optimization should be out['logits_all'] = z_qk # for optimization ? Or I took it wrong?

Thank you for pointing it out! I removed cfvqasimple.py as it is unused and redundant. The cfvqasimple option has been implemented using the cfvqa.py file by setting is_va=False.

yuleiniu commented 3 years ago

Hello, I want to ask a question. Can you get the result in paper, I got very low result in all files of .yaml; And I didn't modify any parameter.

Hi Han, would you like to share more details?

hannajiang commented 3 years ago

Thanks for your reply, I run vqa2/smrl_baseline.yaml get best eval accuracy_top1 is 53.97; run vqacp2/smrl_cfvqa_sum.yaml get beset eval accuracy_cfvqa_top1 is 47.60; I didn't modify any parameters; Maybe I did something wrong?

yuleiniu commented 3 years ago

Thanks for your reply, I run vqa2/smrl_baseline.yaml get best eval accuracy_top1 is 53.97; run vqacp2/smrl_cfvqa_sum.yaml get beset eval accuracy_cfvqa_top1 is 47.60; I didn't modify any parameters; Maybe I did something wrong?

Are the results reported from logs.txt? The results should be reported in the json files. For example, results for CF-VQA can be found in _cfvqa_val_oe.json.

hannajiang commented 3 years ago

Sorry to bother you again, I run the code, and only generated logs.json.tmp but didn't generate the _cfvqa_val_oe.json file.

yuleiniu commented 3 years ago

Sorry to bother you again, I run the code, and only generated logs.json.tmp but didn't generate the _cfvqa_val_oe.json file.

Were there any errors or warnings in the logs? Also, since this repo is built on top of RUBi, are you able to run their code?

hannajiang commented 3 years ago

Yes, there is a ModuleNotFoundError: No module named 'block.external', I have modified the error.Thanks a lot.