search

face-analysis / emonet

Official implementation of the paper "Estimation of continuous valence and arousal levels from faces in naturalistic conditions", Antoine Toisoul, Jean Kossaifi, Adrian Bulat, Georgios Tzimiropoulos and Maja Pantic, Nature Machine Intelligence, 2021
https://www.nature.com/articles/s42256-020-00280-0
Other
260 stars 62 forks source link

about the facial landmarks #13

Open lhr-30 opened 2 years ago

lhr-30 commented 2 years ago

Why does the training set data contain facial landmarks? It seems that the facial landmarks are not used during the training process, just helps to crop the image?

lhr-30 commented 2 years ago

Why does the training set data contain facial landmarks? It seems that the facial landmarks are not used during the training process, just helps to crop the image?

and another question is how do you get the facial landmarks cause i found facial landmarks data in the training set?

HEzelda commented 2 years ago

same here!

lhr-30 commented 2 years ago

Why does the training set data contain facial landmarks? It seems that the facial landmarks are not used during the training process, just helps to crop the image?

and about landmarks, the paper does not menssion the loss function about the landmarks, is there any loss function about the landmarks learning?

ravikiranrao commented 2 years ago

@lhr-30 @HEzelda As far as I understand, this paper does not train landmarks, rather obtain the landmarks from the pre-trained FAN network. The landmarks obtained are thus treated as a heatmap for the attention map.

lhr-30 commented 2 years ago

@lhr-30 @HEzelda As far as I understand, this paper does not train landmarks, rather obtain the landmarks from the pre-trained FAN network. The landmarks obtained are thus treated as a heatmap for the attention map.

But how to get the pre-trained FAN network? Do you mean we get the hyperparameter of the pre-trained FAN network in another way but not in this trainning process?

ravikiranrao commented 2 years ago

But how to get the pre-trained FAN network? Do you mean we get the hyperparameter of the pre-trained FAN network in another way but not in this trainning process?

@lhr-30 Technically, yes. This paper effectively trains only the green part in Fig. 3. The FAN network in which authors refer in the paper is from this.

lhr-30 commented 2 years ago

Now I have difficulty in reproducing this paper. Thr problem is that how to train the FAN part. In the emonet, we only get 'heatmap' but if we want to add the loss function we need to get the face landmarks. So how to transfrom the heatmap to face landmarks or do you have some other way to compute this part of loss function? Thank you.

------------------ 原始邮件 ------------------ 发件人: "face-analysis/emonet" @.>; 发送时间: 2022年6月29日(星期三) 下午5:13 @.>; @.**@.>; 主题: Re: [face-analysis/emonet] about the facial landmarks (Issue #13)

But how to get the pre-trained FAN network? Do you mean we get the hyperparameter of the pre-trained FAN network in another way but not in this trainning process?

@lhr-30 Technically, yes. This paper effectively trains only the green part in Fig. 3. The FAN network in which authors refer in the paper is from this.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

ravikiranrao commented 2 years ago

Sorry @lhr-30, I do not know much about optimizing the FAN yet. As a trivial suggestion, you can contact the authors directly.

Edit: I believe this is the official repo of FAN, may be you can get the landmarks from here.

sssmost commented 1 year ago

Any advance from this: how obtain facial landmarks from heatmap? The paper doesn`t report details about that.