search

chuanyangjin / MMToM-QA

[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering
https://arxiv.org/abs/2401.08743
MIT License
111 stars 14 forks source link

✅ `inference_raw.json` for `BIP-ALM` test evaluation in reproducing #4

Open chunhuizng opened 2 months ago

chunhuizng commented 2 months ago

Hi Chuanyang,

Could you release the inference_raw.json file used in text_parsing.py for test evaluation?

To reproduce the _BIP-ALM_method test, we need the multimodal_representations.json file. However, generating this file requires running text_parsing.py with inference_raw.json, which is not currently available in the repo (and also lacks instructions on how to create it).

If the inference_raw.json file can be released (or, more straightforwardly, if the final evaluation test data multimodal_representations.json for BIP-ALM can be released), it will enable successfully reproducing the test evaluation of the proposed BIP-ALM method. This would help to enhance the impact and reproducibility of this smart paper within the community.

Thank you very much!

chuanyangjin commented 2 months ago

We have released the requested files. You can now find them in the folder "files" for use in reproducing our results. Thank you for bringing this to our attention!

chunhuizng commented 2 months ago

We have released the requested files. You can now find them in the folder "files" for use in reproducing our results. Thank you for bringing this to our attention!

Thank you very much, Chuanyang! They are super helpful.

I would also like to share some additional .json files used in the middle process: Google Drive folder in case anyone wants to use them.