enable MARLIN for emotion recognition on the ravdess and celebvhq dataset

ControlNet / MARLIN

[CVPR] MARLIN: Masked Autoencoder for facial video Representation LearnINg

https://openaccess.thecvf.com/content/CVPR2023/html/Cai_MARLIN_Masked_Autoencoder_for_Facial_Video_Representation_LearnINg_CVPR_2023_paper

Other

209 stars 20 forks source link

enable MARLIN for emotion recognition on the ravdess and celebvhq dataset #24

Open aromanusc opened 5 months ago

ControlNet commented 5 months ago

Thank you for your contribution. I think the changes are too destructive for other tasks now. I will work on the merging.

aromanusc commented 5 months ago

Hey, thanks for the prompt reply. I meant to put this as a draft, I agree the changes are not in the best condition to be merged. I can maybe add a generic dataset loader under /dataset. I also have to remove some loose comments I placed mainly for diff purposes.

I have to say that for the emotion recognition task on the RAVDESS dataset, MARLIN performs great! Now I am working towards an audio-visual implementation with MARLIN. Thanks for keeping this repo in such a great shape.

ControlNet commented 5 months ago

Oh my mistake. I thought the pr is for emotion recogntiion for CelebV-HQ. now I have modified the code to make it workable for emotion task on CelebV-HQ.

It will be great if you can prepare the code for RAVDESS with the new python files dataset/ravdess.py, preprocess/ravdess_preprocess.py and preprocess/ravdess_extract.py, and modify the evaluate.py?