How to perform Head Parsing?

[snehith-r]

In the paper you have mentioned to use HPBTT on the fly to extract the head region, but the method initially estimates the SMPL mesh using hmr and then used the smpl part segmentation to generate the part maps, given that SMPL estimated is not completely aligned with input image always, how did you extract the entire face region to perform the quantitative comparison.

TIA

[yukangcao]

Hi, thanks for your question. You raise a good point about the alignment

• It's better to have a roughly well-aligned SMPL estimation. You can do that robustly with 1. multi-view fitting, and 2. optimization with pytorch3d's differentiable rendering

• For SMPL parsing, generally, we have 24 parts at most, which could be viewed as Each vertex from the SMPL model has its own label indicating which part it belongs to.

• Then, say we have the SMPL and ground-truth mesh to be roughly aligned, for example: where the blue one is the SMPL model, and red represents the ground-truth mesh For each point in the ground-truth mesh, we find its closest SMPL vertex and assign the corresponding parsing label. If our target is the head, we will get:

Hope this helps