wang-zidu
commented
1 month ago Thank you for your interest. $Cp$ is the target, obtained from real faces. We also need to obtain the predicted $V{2d}^p$ corresponding to target $Cp$. Due to the topological consistency of BFM, we only need masks (annotations) to get $V{2d}^p$. However, before 3DDFA-V3, there were no 3D mesh masks consistent with the 2D face segmentation regions, so some parts that confused you (such as Algorithm 1 in the supplementary materials) were involved in the calculation of these 3D mesh masks. These part ultimately resulted in the 3D mesh segmentation indices as shown in the figure below:
These parts merely show the steps we took to obtain the masks (indices, annotations). We didn't use manual annotation directly; instead, we calculated these masks using a 2D projection method and then made manual corrections. If you believe that our segmented annotation regions of BFM are consistent with the semantics of the 2D segmentation, there's no need for further annotation. You can directly use model['annotation'] and model['annotation_tri'] in /assets/ . If you need to view the results in .obj format, you can download meanshape-seg.obj from /assets/.
haojc
Thanks for your excellent work! I have a question about the symbol $C_p$ in the paper. It seems to indicate two meanings. In subsection Transforming Segmentation to 2D points, the segmentation of each pixel $C_p$ is obtained based on the raw RGB image via a segmentation method. However, in Algorithm 1 of Supplementary Material, $Cp$ is obtained based on the rendered image $I{seg}=Seg(Render(V_{3d},Tex)$. I wonder, (1) whether these two $C_p$ indicate the same segmentation results? If so, which image is selected, the raw image, or the rendered one? (2) since we already obtained the pixel segmentation based on the raw RGB image, why we render an image from 3d mesh when determining the vertex segmentation annotation? Thanks.