search

cleardusk / 3DDFA_V2

The official PyTorch implementation of Towards Fast, Accurate and Stable 3D Dense Face Alignment, ECCV 2020.
MIT License
2.82k stars 505 forks source link

what makes such lightweight backbone works so well? #23

Open makpia opened 3 years ago

makpia commented 3 years ago

compared to the previous version of your work, 3ddfa, 3ddfa_v2's structure is much simpler, but achieves better results. so i wonder if the meta-joint loss is the reason that enable mobilenet to outperform previous works. i would like to know your opinion on applying these methods(look ahead, combine different losses) to solving other tasks.

chengfeng0113 commented 3 years ago

If it's not for the data, it can even be done more finely, and this method is the best solution so far.

wjzlwz commented 3 years ago

If it's not for the data, it can even be done more finely, and this method is the best solution so far.

so,how we can generate high quality data for better result?fix the landmarks error in 300w data(like eyes) or rebuild a 3dmm model to replace bfm model?

chengfeng0113 commented 3 years ago

If it's not for the data, it can even be done more finely, and this method is the best solution so far.

so,how we can generate high quality data for better result?fix the landmarks error in 300w data(like eyes) or rebuild a 3dmm model to replace bfm model?

You can improve from the following directions:

  1. Use better face shape models and expression models, such as BFM2019, but you need to find the keypoints of the face from many vertices. I use the BFM2009 model.
  2. Use better 2D keypoint models.
  3. Use a better 3DMM model, I use the 3DDFA model.
  4. Consider the face data balance of various Euler angles when making the face profile.
  5. To solve the problem of poor performance with eyes open and closed, you can consider using StyleGAN2 and InterFacegan to synthesize face data with eyes closed.
lu17 commented 3 years ago

I used resnet50 instead of mobilenet_V1, but strangely, the results were worse. Why is such a simple model better?

shoutOutYangJie commented 3 years ago

@chengfeng0113 what is BFM model? I am newly attached to this field. And How should I do to systhesis short video like 3ddfa_v2 paper?

chengfeng0113 commented 3 years ago

@chengfeng0113 what is BFM model? I am newly attached to this field. And How should I do to systhesis short video like 3ddfa_v2 paper? Regarding the BFM model, you can refer to BFM. Oh ha, I also want to know how to synthesize short videos online, I can only use FaceProfiling to synthesize faces with continuous changes in Euler angles offline.

shoutOutYangJie commented 3 years ago

@chengfeng0113 so after face profiling, the image boundary appears "zero" pixel ? I notice that in 3ddfaV1, face profiling will make "zero" pixel at image boundary. But in 3ddfav2, as author shows, there is no "zero" pixel.

shoutOutYangJie commented 3 years ago

@chengfeng0113 and do you know what is "tri"? I am not familiar with this field. thank you.

ghost commented 3 years ago

do you know how to generate the 3D render picture with color? thanks. @chengfeng0113

liyuanyaun commented 2 years ago

i have try resnet-22 for reconstruction, but it's worse than your gif which put on the github? especially the mouth always open. can you tell me why? if you did something different from you public data?

liyuanyaun commented 2 years ago

i have try resnet-22 for reconstruction, but it's worse than your gif which put on the github? especially the mouth always open. can you tell me why? if you did something different from you public data?

@cleardusk