search

SimonGiebenhain / NPHM

[CVPR'23] Learning Neural Parametric Head Models
https://simongiebenhain.github.io/NPHM/
Other
211 stars 17 forks source link

How would you do the inverse fitting process (from Neural Parametric Head Models to photorealistic portrait) #3

Open oijoijcoiejoijce opened 1 year ago

oijoijcoiejoijce commented 1 year ago

Super cool implementation, thanks for that! How would you go about the inverse fitting process (from Neural Parametric Head Models to photorealistic portrait)?

SimonGiebenhain commented 1 year ago

I am not exactly sure what you mean. Are you interested in generating realistic images from a fitted NPHM?

If that is the case, NPHM currently does not support any sort of appearance modelling.

However, in the ideal scenario the fitted NPHM is able to capture the persons shape and expressions accurately, e.g. throughout a video. In such a scenario one could try to use NeRFace in order to model the appearance of a person (i.e. replace FLAME parameters with NPHM parameters that are fed to NeRFace).