Any reference about extract new texture from new mask?

[luameows]

Thks for ur excellent project, and the 3d face-mask is very robust. However, I want add more kinds of mask texture for face-mask generation but cannot found any documention about how to do this. Can u provide some information?

[wang21jun]

Thanks for your interest in our project. Please refer to our tech-report, In short, given a masked face image R, we first compute the uv position map P by PRNet. Then we obtain the 'uv_mask.png' by map the segmentation map of R into uv space with the uv position map P.

[fengshikun]

@wang21jun , thanks for your explanation, still i have some questions, i notice that the mask texture file under path Data/mask-data(called 0.png, 1.png...) is a 4-channel matrix. 1.Is that 0-3 channel of this matrix(ref_texture_src) is the uv_texture generated by PRNet by keeping the mask part of original image using the mask segmentation?
2.What' s the meaning of 4th channel of this 4-channel matrix(uv_mask_src), how to generate this channel?

[lucia123]

@wang21jun , thanks for your explanation, still i have some questions, i notice that the mask texture file under path Data/mask-data(called 0.png, 1.png...) is a 4-channel matrix. 1.Is that 0-3 channel of this matrix(ref_texture_src) is the uv_texture generated by PRNet by keeping the mask part of original image using the mask segmentation? 2.What' s the meaning of 4th channel of this 4-channel matrix(uv_mask_src), how to generate this channel?

The 4th channel is the alpha channel of RGBA image. It can be obtained by the segmentation model or Photoshop tool. The values in this channel mean the probability that pixels belong to mask region.

[AlonZolfi]

Thanks for your interest in our project. Please refer to our tech-report, In short, given a masked face image R, we first compute the uv position map P by PRNet. Then we obtain the 'uv_mask.png' by map the segmentation map of R into uv space with the uv position map P.

Could you please elaborate more on how did you do the 3d reconstruction on the mask only (step (b) -> (c))?

[Phi-C]

@AlonZolfi The pseudo code is as follows

def get_mask_texture(src_image_path, seg_image_path):
    seg_image = imread(seg_image_path)
    src_image = imread(src_image_path)

    # use src_image to get pos and vertices with the help of PRN model
    lms_info = read_info.read_landmark_106_array(src_face_lms)
    pos = prn_model.process(src_image, lms_info)
    vertices = prn_model.get_vertices(pos)

    # map mask texture to UV texture map
    mask_image = mask_image/255.
    mask_UV = cv2.remap(mask_image, pos[:, :, :2].astype(np.float32), None, interpolation=cv2.INTER_NEAREST, boarderMode=cv2.BORDER_CONSTANT, borderValue=(0))
    mask_UV = mask_UV.astype(np.float32)
    mask_UV = cv2.cvtColor(mask_UV, cv2.COLOR_RGB2RGBA)
    mask_UV[background_pos, 3]  = 0
    return mask_UV

[liujia761]

感谢您对我们项目的关注。请参考我们的技术报告，简而言之，给定一个蒙版面部图像R，我们首先通过PRNet计算UV位置图P。然后，我们将R的分割图映射到UV空间，并使用UV位置图P获得“uv_mask.png”。

您能否详细说明您是如何仅在面具上进行 3D 重建的（步骤 （b） -> （c））？

好喽 您这个问题解决了吗，我在用这个的时候也遇到了这个问题

[liujia761]

@AlonZolfi伪代码如下

def get_mask_texture(src_image_path, seg_image_path):
    seg_image = imread(seg_image_path)
    src_image = imread(src_image_path)

    # use src_image to get pos and vertices with the help of PRN model
    lms_info = read_info.read_landmark_106_array(src_face_lms)
    pos = prn_model.process(src_image, lms_info)
    vertices = prn_model.get_vertices(pos)

    # map mask texture to UV texture map
    mask_image = mask_image/255.
    mask_UV = cv2.remap(mask_image, pos[:, :, :2].astype(np.float32), None, interpolation=cv2.INTER_NEAREST, boarderMode=cv2.BORDER_CONSTANT, borderValue=(0))
    mask_UV = mask_UV.astype(np.float32)
    mask_UV = cv2.cvtColor(mask_UV, cv2.COLOR_RGB2RGBA)
    mask_UV[background_pos, 3]  = 0
    return mask_UV

您好，请问这个函数返回值是从（a）上取下口罩返回的（c）吗 另外这个函数的输入是什么呢

[liujia761]

感谢您的出色项目，3d 面罩非常坚固。 但是，我想为面罩生成添加更多种类的面罩纹理，但找不到有关如何执行此操作的任何文档。你能提供一些信息吗？ 好喽，大佬请问您这个问题解决了吗，我也遇到了这个问题

[liujia761]

感谢您的出色项目，3d 面罩非常坚固。 但是，我想为面罩生成添加更多种类的面罩纹理，但找不到有关如何执行此操作的任何文档。你能提供一些信息吗？

您好。大佬您这个问题就解决了吗，我也遇到了这个问题