search

chuanyangjin / MMToM-QA

[🏆Outstanding Paper Award at ACL 2024] MMToM-QA: Multimodal Theory of Mind Question Answering
https://arxiv.org/abs/2401.08743
MIT License
115 stars 14 forks source link

`finetuning_data`✅, `TEST_INDICES`✅, and `training video`❓ Release #2

Open chunhuizng opened 3 months ago

chunhuizng commented 3 months ago

Could you release the TEST_INDICES and finetuning_data used in graph_utils.py and generate_finetuning_dataset.py?

Thanks!

chuanyangjin commented 3 months ago

Thanks for pointing this out! We've released the TEST_INDICES in graph_utils.py. We will release the finetuning_data soon.

chunhuizng commented 3 months ago

Thanks for pointing this out! We've released the TEST_INDICES in graph_utils.py. We will release the finetuning_data soon.

Thank you and very look forward to it!

chunhuizng commented 3 months ago

Thanks for kindly sharing the training_set.zip!

By the way, could you also share the video data in the training_set.zip?

The current version includes belief_{num}.pik, camera_data_{num}.pik, graph_{num}.pik, init_graph.pik, instanace_colors.pik, env_info.pik, actions.pik.

if training_set.zip can include the vision data, such as video-related RGB and depth file, it would be helpful to reproduce and conduct multimodal training/finetuning experiments (I guess the path is in task_{episode}/script/0/).

Thanks!