Question about the optimization stage?

[lincoln1128]

Dear author, thank you for sharing the awesome code! I have tried using the code, but I found the generated video of the visibility estimation stage is even better than the video obtained of the final optimization stage, especially for the textures of the visible parts. This result is not difficult to understand, because the optimization goal of the second stage is actually the output of the first stage, so it can be considered that the upper bound of the second stage is actually the output of the first stage. So my question is, what is the motivation of the design of the optimization stage?

[ken-ouyang]

Hi, lincoln1128. For novel view synthesis, it is possible to stop at the second stage to achieve general good results. The reason that why we need the optimization stage is to retrieve the optimized latent for the attribute editing. Without this stage, it is impossible to apply the desired latent direction (e.g., smile, ages etc. ).

[lincoln1128]

My problem solved.

[luchaoqi]

I got a follow-up question - the inversion pipeline from recent SOTA papers (PTI/Mystyle/HFGI3D) usually consists of 2 stages: 1. update latent code (w/w+) 2. update the model based on their own strategies If I understand the code correctly here, HFGI3D doesn't change stage 1. It basically inherits the same w_projector/w_plus_projector function from PTI and does model tuning in stage 2 based on the latent code from stage 1, right?