search

jingyang2017 / emonet_train

Emonet unofficial Implemented "Estimation of continuous valence and arousal levels from faces in naturalistic conditions" published in Nature Machine Intelligence 2021
MIT License
18 stars 3 forks source link

the loss function #1

Closed lhr-30 closed 2 years ago

lhr-30 commented 2 years ago

I'm reproducing this paper now and I have some trouble training this net. I found that I wrote the same loss function to yours, but the result seems that the prediction of the heatmap did not get trained. But in your code there is also no loss function about the heatmap and FAN part. So I wonder how it works? thanks

jingyang2017 commented 2 years ago

I'm reproducing this paper now and I have some trouble training this net. I found that I wrote the same loss function to yours, but the result seems that the prediction of the heatmap did not get trained. But in your code there is also no loss function about the heatmap and FAN part. So I wonder how it works? thanks

From my understanding, the feature maps from FAN is seen as the input to a learnable network. FAN part is pretrained on face alignment datasets and is fixed in emotion recognition task. FAN functions as a feature provider.

lhr-30 commented 2 years ago

Can you tell me where you get the pretrained hyperparameter of the FAN? I tried to find it but didn't get it?

jingyang2017 commented 2 years ago

Can you tell me where you get the pretrained hyperparameter of the FAN? I tried to find it but didn't get it?

You can extracted the FAN pretrained parameters from https://github.com/face-analysis/emonet/tree/master/pretrained. Or you can use https://github.com/hhj1897/fan_training/tree/master/pretrained

lhr-30 commented 2 years ago

Thanks for your help, now I can run the FAN part correctly. But I still get a result not good as the paper said. I trained my network on th AffectNet dataset with 26w pictures to train the 8-classes net.I only got 60% ACC and PCC、CCC results were also not that good. Can you tell me your results? Are there any other data augmentation methods used?

jingyang2017 commented 2 years ago

Thanks for your help, now I can run the FAN part correctly. But I still get a result not good as the paper said. I trained my network on th AffectNet dataset with 26w pictures to train the 8-classes net.I only got 60% ACC and PCC、CCC results were also not that good. Can you tell me your results? Are there any other data augmentation methods used?

I have put my trained model in https://github.com/jingyang2017/emonet_test/tree/master/ibug/emotion_recognition/emonet/weights. I get very close results as the paper said.