UV coordinates for BFM09?

[okdalto]

Hello there!

First of all, I want to express my gratitude for your excellent work.

I have a question regarding the use of UV coordinates for BFM 09 models.

I have found a resource for UVs at https://github.com/anilbas/3DMMasSTN/blob/master/util/BFM_UV.mat, but I have encountered an issue.

The vertex numbers in the BFM 09 model and the UV resource don't seem to match.

I was wondering if you or anyone else has any suggestions on how to resolve this issue?

[ascust]

Hi, I have been busy and did have time to reply. About the vertex number, how many are there in the UV map you found? I suggest you could refer to file "convert_bfm09_data.py", where the original BFM09 is converted to the one used in this project. The reason why we have to convert the original model is that the expression bases are from another project that do not match the shape bases. So there are some small modification. I guess you could just take a look at how the vertex indices are selected and you may do the same for the UV map?

[okdalto]

Thank you for kind comment! I will try to look at the code and let you know!

[okdalto]

For those who encountered the same issue that I have been through, I'm leaving the code to save BFM09_model_info.mat with UV coordinate provided here. Just download the mat in the BFM folder and run the code. :)

from scipy.io import loadmat, savemat
import numpy as np
from array import array

# load expression basis

def LoadExpBasis():
    n_vertex = 53215
    Expbin = open('BFM/Exp_Pca.bin', 'rb')
    exp_dim = array('i')
    exp_dim.fromfile(Expbin, 1)
    expMU = array('f')
    expPC = array('f')
    expMU.fromfile(Expbin, 3*n_vertex)
    expPC.fromfile(Expbin, 3*exp_dim[0]*n_vertex)

    expPC = np.array(expPC)
    expPC = np.reshape(expPC, [exp_dim[0], -1])
    expPC = np.transpose(expPC)

    expEV = np.loadtxt('BFM/std_exp.txt')

    return expPC, expEV

# transfer original BFM09 to our face model

def transferBFM09():
    original_BFM = loadmat('BFM/01_MorphableModel.mat')
    shapePC = original_BFM['shapePC']  # shape basis
    shapeEV = original_BFM['shapeEV']  # corresponding eigen value
    shapeMU = original_BFM['shapeMU']  # mean face
    texPC = original_BFM['texPC']  # texture basis
    texEV = original_BFM['texEV']  # eigen value
    texMU = original_BFM['texMU']  # mean texture

    expPC, expEV = LoadExpBasis()

    # transfer BFM09 to our face model

    idBase = shapePC*np.reshape(shapeEV, [-1, 199])
    idBase = idBase/1e5  # unify the scale to decimeter
    idBase = idBase[:, :80]  # use only first 80 basis

    exBase = expPC*np.reshape(expEV, [-1, 79])
    exBase = exBase/1e5  # unify the scale to decimeter
    exBase = exBase[:, :64]  # use only first 64 basis

    texBase = texPC*np.reshape(texEV, [-1, 199])
    texBase = texBase[:, :80]  # use only first 80 basis

    # our face model is cropped align face landmarks which contains only 35709 vertex.
    # original BFM09 contains 53490 vertex, and expression basis provided by JuYong contains 53215 vertex.
    # thus we select corresponding vertex to get our face model.

    index_exp = loadmat('BFM/BFM_front_idx.mat')
    index_exp = index_exp['idx'].astype(
        np.int32) - 1  # starts from 0 (to 53215)

    index_shape = loadmat('BFM/BFM_exp_idx.mat')
    index_shape = index_shape['trimIndex'].astype(
        np.int32) - 1  # starts from 0 (to 53490)
    index_shape = index_shape[index_exp]

    idBase = np.reshape(idBase, [-1, 3, 80])
    idBase = idBase[index_shape, :, :]
    idBase = np.reshape(idBase, [-1, 80])

    texBase = np.reshape(texBase, [-1, 3, 80])
    texBase = texBase[index_shape, :, :]
    texBase = np.reshape(texBase, [-1, 80])

    exBase = np.reshape(exBase, [-1, 3, 64])
    exBase = exBase[index_exp, :, :]
    exBase = np.reshape(exBase, [-1, 64])

    meanshape = np.reshape(shapeMU, [-1, 3])/1e5
    meanshape = meanshape[index_shape, :]
    meanshape = np.reshape(meanshape, [1, -1])

    uv = loadmat('BFM/BFM_UV.mat')["UV"]
    uv = np.reshape(shapeMU, [-1, 3])
    uv = uv[index_shape, :]
    uv = np.reshape(meanshape, [1, -1])

    meantex = np.reshape(texMU, [-1, 3])
    meantex = meantex[index_shape, :]
    meantex = np.reshape(meantex, [1, -1])

    # other info contains triangles, region used for computing photometric loss,
    # region used for skin texture regularization, and 68 landmarks index etc.
    other_info = loadmat('BFM/facemodel_info.mat')
    frontmask2_idx = other_info['frontmask2_idx']
    skinmask = other_info['skinmask']
    keypoints = other_info['keypoints']
    point_buf = other_info['point_buf']
    tri = other_info['tri']
    tri_mask2 = other_info['tri_mask2']

    # save our face model
    savemat('BFM/BFM09_model_info.mat', {'meanshape': meanshape, 'meantex': meantex, 'idBase': idBase, 'exBase': exBase, 'texBase': texBase,
                                         'tri': tri, 'point_buf': point_buf, 'tri_mask2': tri_mask2, 'keypoints': keypoints, 'frontmask2_idx': frontmask2_idx, 'skinmask': skinmask, "uv": uv})

if __name__ == '__main__':
    transferBFM09()