search

katerynaCh / MMA-DFER

This repository provides the codes for MMA-DFER: multimodal (audiovisual) emotion recognition method. This is an official implementation for the paper MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild.
13 stars 1 forks source link

Bad results #4

Closed ybh-yw closed 2 weeks ago

ybh-yw commented 3 months ago

Why do I get poor results after running the same script file? And I used the DFEW dataset as well as the WAV audio converted by the software as the input data.

katerynaCh commented 3 months ago

Hi,

can you provide more details? Are you trying to evaluate provided models or doing training? What WAR/UAR are you getting? How did you extract wav files? Are you using the pre-trained MAEs for audio and vision branches (as mentioned in README)?

ybh-yw commented 3 months ago

I only trained dfew and got a set of results that UAR: 38.00 WAR: 45.04 .I did load two pth files and then I didn't know how to extract the wav file so I used the conversion class software to extract the original wav audio. Can you give some advices Thank you

katerynaCh commented 2 months ago

I have added a script to extract audio from mp4s with ffmpeg here, can you try to extract the audio following it? You will need to put all raw videos into one folder 'raw', and it will extract audios accordingly to 'raw_wav'.

Additionally, you can try to download pre-trained models and evaluate on your data using the evaluation script, if the UAR/WAR is still low then it is likely some issue on the data preprocessing side

katerynaCh commented 2 months ago

Hi @ybh-yw were you able to achieve the desired results with ffmpeg extraction? (Note that other users report being able to reproduce DFEW results so I expect it to be the pre-processing issue)

katerynaCh commented 2 weeks ago

Closing this issue due to inactivity and I believe that correct results should be achieved if you use the provided script for audio extraction. In addition, others have reported no problem reproducing DFEW. Feel free to reopen if you need more help with this.