search

VT-NLP / Mocheg

Dataset and Code for Multimodal Fact Checking and Explanation Generation (Mocheg)
Apache License 2.0
36 stars 8 forks source link

Verification Model's Checkpoints Other Than Text-Image-Evidence Cases #11

Open given131 opened 6 months ago

given131 commented 6 months ago

Hello!

I'm running experiments based on Mocheg. I found that the implementation offers various options for the model (https://github.com/VT-NLP/Mocheg/blob/main/verification/util/enums.py#L11), it seems that checkpoint is only offered for CLAIM_TEXT_IMAGE_attention_5_4

I'm currently dealing with a case where there exists image evidence only. I have tried CLAIM_IMAGE_attention_5_4 option, and found that it internally goes through a classifier network named classifier_text. (https://github.com/VT-NLP/Mocheg/blob/main/verification/util/stance_detect.py#L217) I'm a bit confused due to the name because it sounds like the module is basically for the text-evidence-only model.

Is it that classifier_text is actually for the single-modality evidence regardless of modality (i.e., whether it is image-only or text-only?) Or is there other checkpoint for image-only case? If so, could you please kindly share the checkpoint of other options as well including image-evidence-only case?

Thank you.