Why is answers set to 0 for irtr even for the positive case?

dandelin / ViLT

Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"

Apache License 2.0

1.36k stars 209 forks source link

Closed TheShadow29 closed 2 years ago

TheShadow29 commented 2 years ago

Hello, thanks for the amazing repository. If I understand correct, for IRTR, answer should be 1 for the first element which is true, and 0 for the remaining false texts (https://github.com/dandelin/ViLT/blob/master/vilt/modules/objectives.py#L429)

But in the code, it sets all of them including the positive sample to be zero. Am I missing something here? Thanks!

TheShadow29 commented 2 years ago

Whoops, I just realized it is cross entropy, not binary cross entropy. Please ignore the issue.

mactavish91 commented 1 year ago

@TheShadow29 So why is answers always set to 0? All tags are 0.

TheShadow29 commented 1 year ago

@mactavish91 first ITM is true, others are false. So targets are all 0.