search

AutoHDR / HD-Net

3 stars 0 forks source link

didn't see the part of getting unwarped texture as the input of ASD-Net? #1

Open anewusername77 opened 1 year ago

anewusername77 commented 1 year ago

Thanks for your work! I didn't find the code for, as said in your paper, using Nonlinear 3DMM to unwarp face images for alignment?

AutoHDR commented 1 year ago

Thank you for your comments. We did not release the unwarping and warping because it borrows from Nonlinear 3DMM(CVPR'18). You can use their pre-trained model for your unwarping and warping. By the way, our work for this paper began in 2018 and took a long time for publication, there is a lot of follow-up work that can reduce the loss of unwarping and warping, such as Nonlinear 3DMM(CVPR'19), DECA + Piecewise Affine Transformation. Thus, we suggest that you can try the follow-up works for unwarping and warping.

anewusername77 commented 1 year ago

thanks a lot! I'll try Nonlinear 3DMM.

anewusername77 commented 1 year ago

Hi, sorry to bother you again, I tried evaluation code of Nonlinear 3DMM(CVPR'19) on AFLW2000-3D, but the result seems weird(for example, face of an input image dosen't open mouth but the output of Nonlinear 3DMM opens)

AutoHDR commented 1 year ago

yes, face image to 2D texture space is a hard problem to deal with the twisted parts. Some strange parts will appear due to incorrect face parameter estimation and the error of the mapping function. Thus, Nonlinear 3DMM also meets the same problems that lead to mouth open(e.g. Figure 9 Recon.). In our paper, in order to compare with other methods, we use Poisson blending [31] to rebuild relit faces with backgrounds(e.g. Figure 9 Relit).

anewusername77 commented 1 year ago

Thanks for your reply, Just to make sure, the input of your network should be unwarped images?(using Nonlinear 3DMM and saving fitted unwarped texture images as dataset, then directly loading them as input while training).

One more thing, it seems like reconstructed images of Nonlinear 3DMM's pretrain-model can't achieve as good results as figure 3(e) or figure 9. I tested on AFLW2000 and it's like a fake face(no personal identities at all)...

AutoHDR commented 1 year ago

yes, we consider that the unwarped texture face images are alignment and use the aligned images for real face image training.

What is the quality of the sample provided that you run their code? In my experience, the unwarped texture images will lose relatively small details, and the details will be lost when restoring to the original face images from textures.