search

yerfor / Real3DPortrait

Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis; ICLR 2024 Spotlight; Official code
MIT License
967 stars 112 forks source link

Network architecture of I2P and MA modules #29

Closed trThanhnguyen closed 7 months ago

trThanhnguyen commented 9 months ago

Hi, authors. Thanks for sharing your work. Can you explain the reason for the design of the I2P and MA modules? The MA looks like a ViT branch in the I2P, why it does not have the VGG branch as well? image

yerfor commented 7 months ago

Hi,

The I2P model aims at reconstruting a accurate 3D face representation with rich geometry and texture details, so we use a hybrid artecture of VGG and ViT. I2P is a large model that have >80M parameters.

By contrast, the MA only aims at learning the geometry difference needed to control the expression, so we use an efficient archtecture, SegFormer-b0, which is parameter-light.