search

LizhenWangT / FaceVerse

FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset (CVPR2022)
BSD 2-Clause "Simplified" License
460 stars 57 forks source link

Render mesh in static position? #11

Closed JamesBrod closed 2 years ago

JamesBrod commented 2 years ago

Hi again,

I am currently trying to use the offline tracking to render the face mesh in a static position and rotation while processing a video. So ideally I want the face to remain in the same location, and simply change expressions, rather than following the face movement and rotation. Do you have any advice on how I could implement this or where in the current code I could start looking and adjusting? I've been going through the different scripts, but some advice would be greatly appreciated. Thanks!

LizhenWangT commented 2 years ago

This can be done by just using a fixed trans_tensor and rot_tensor in model/FaceVerse.py. For example

id_coeff, exp_coeff, tex_coeff, angles, gamma, translation = faceverse_model.split_coeffs(faceverse_model.get_packed_tensors())
# use a fixed rot and trans here
angles_fixed = the rotation that you need
translation_fixed = the translatetion that you need
render_coeff = faceverse_model.merge_coeffs(id_coeff, exp_coeff, tex_coeff, angles_fixed, gamma, translation_fixed )
pred_dict = faceverse_model(render_coeff , render=True, texture=True)
rendered_img_c = pred_dict['rendered_img']
rendered_img_c = np.clip(rendered_img_c.cpu().numpy(), 0, 255)
pred_dict = faceverse_model(render_coeff, render=True, texture=False)
rendered_img_r = pred_dict['rendered_img']
rendered_img_r = np.clip(rendered_img_r.cpu().numpy(), 0, 255)
JamesBrod commented 2 years ago

Thanks so much again, I've managed to get it working. I tried using a fixed translation and rotation tensor in the forward method before and was getting some very strange results, but now it's all good!