search

NUSTM / FacialMMT

Code for paper "A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations"
GNU General Public License v3.0
55 stars 3 forks source link

meld_multimodal_train-T+A+V_bert large. dt #7

Closed Yang-Liz closed 7 months ago

Yang-Liz commented 7 months ago

In the data reading function, it is necessary to call 'FacialMMT main/reprocess_data/T+A+V/meld_multimodal_train-T+A+V_bert large. dt'. How should this file be generated?

wjzhengnlp commented 7 months ago

The following code will generate the.dt file you need:

https://github.com/NUSTM/FacialMMT/blob/main/utils/util.py#L90

Yang-Liz commented 7 months ago

I have taken note of your advice regarding the generation of the ".dt" file using the "meld_text_path" file. However, I am uncertain about how to obtain the "meld_text_path" file with the specified path '/media/devin/data/meld/preprocess_data/ref/update'. Could you please provide some guidance on how to obtain or generate the "meld_text_path" file? Specifically, I would like to know:

  1. Is the "meld_text_path" file a part of the dataset provided with the paper, or is it generated separately during the preprocessing stage?
  2. If it is generated separately, could you please provide instructions on how to create or obtain this file, including any preprocessing steps or scripts that may be required?
  3. If it is a part of the provided dataset, could you please specify where I can locate it within the dataset directory structure?
wjzhengnlp commented 7 months ago

train/val/test_text.json are preprocessing files.

More detail about the MELD dataset can be found in https://affective-meld.github.io/.

Yang-Liz commented 7 months ago

Thank you for your reply, but in the https://affective-meld.github.io/ only "MELD. Raw. tar. gz" can be downloaded and decompressed to obtain MELD.Raw, there is still no "/meld/preprocess_data/" file available

wjzhengnlp commented 7 months ago

I don't understand what you mean. These are files that we have already processed ourselves. Are you encountering any problems while using them? You can paste your bug.

Yang-Liz commented 7 months ago

为了生成dt文件需要load多模态数据https://github.com/NUSTM/FacialMMT/blob/8889c5bdbb2c2c282dc87045aed4b2893d86aedb/utils/util.py#L95

在loading_multimodal_data函数中涉及到了args.load_anno_csv_path和args.meld_text_path（https://github.com/NUSTM/FacialMMT/blob/8889c5bdbb2c2c282dc87045aed4b2893d86aedb/utils/dataset.py#L295）。args.load_anno_csv_path是在meld官网可以下载的。args.meld_text_path是指什么啊？

wjzhengnlp commented 7 months ago

https://github.com/NUSTM/FacialMMT/blob/main/main.py#L17

you can download train/val/test_text.json from:

image