search

adobe-research / diffusion-rig

Code Release for DiffusionRig (CVPR 2023)
https://diffusionrig.github.io
Other
262 stars 16 forks source link

Controlling the eye pose #9

Open atgao opened 10 months ago

atgao commented 10 months ago

Inside the inference script, we can control the pose with here. Is it also possible to control or condition on the eye pose since FLAME has controllable eye joints? I tried to modify the eye pose in the decoding step https://github.com/adobe-research/diffusion-rig/blob/d9b266102e1900032efc62e66a7f6c66dde5ff79/decalib/models/FLAME.py#L189 but it appears that the eye pose is still the same

zh-ding commented 10 months ago

Since we utilize DECA to obtain physical buffers but DECA does not output eye_pose parameters, the model could not learn to control the eye pose. FLAME would use a default eye pose if the parameters are not provided. As you can see in the teaser figure, the eye poses of the physical buffers are neutral regardless of the inputs.