search

SimonGiebenhain / NPHM

[CVPR'23] Learning Neural Parametric Head Models
https://simongiebenhain.github.io/NPHM/
Other
236 stars 19 forks source link

Random Expression Sampling #20

Open oneThousand1000 opened 3 months ago

oneThousand1000 commented 3 months ago

Hi! Thanks for sharing your nice work!

I am wondering how to sample meaningful expression latent code. For example, those presented on the "Latent Expression Interpolation" section of your project page.

image

Looking forward to your reply!

SimonGiebenhain commented 3 months ago

Hi oneThousand1000,

for this visualization I took expressions from the expression_codebook from the training set. However, with the NPHM model there are often some unnatural deformation in the lip region.

MonoNPHM provides much better movemnt of the face and is more robust/consistent.

FYI, NPHM models forward deformations (compatible with rasterization) and MonoNPHM represents backward deformations (compatible with NeRF/ray-based rendering).

Are you looking to do anything specific? Let me know what you would need.

Kind regards, Simon

oneThousand1000 commented 3 months ago

Hi oneThousand1000,

for this visualization I took expressions from the expression_codebook from the training set. However, with the NPHM model there are often some unnatural deformation in the lip region.

MonoNPHM provides much better movemnt of the face and is more robust/consistent.

FYI, NPHM models forward deformations (compatible with rasterization) and MonoNPHM represents backward deformations (compatible with NeRF/ray-based rendering).

Are you looking to do anything specific? Let me know what you would need.

Kind regards, Simon

Thank you for your response!

I have tried using MonoNPHM and have two questions:

  1. I processed a video with approximately 240 frames, and it took over 10 hours to complete. Is this processing time typical for MonoNPHM?
  2. Can the MonoNPHM model be fitted to a point cloud, similar to fitting_pointclouds.py from the NPHM repository?

Thank you again!