the m(matrix data) in *_param.dat

[wangxinjun56]

hi, could you tell me where are the m(matrix data) and shape datas come from ?

acording the paper says "With the face profiling technique, Zhu et al. [33] expand the 300W dataset [68] into 122, 450 images with fitted 3DMM shapes and projection parameters m."

but i can't find it, could you give me the accuracy network link?

[tranluan]

http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/main.htm

[wangxinjun56]

thanks , i get the link, but there say 7 data:

%% Pose Parameter phi = Pose_Para(1); % pitch gamma = Pose_Para(2); % yaw theta = Pose_Para(3); % roll t3dx = Pose_Para(4); % translation t3dy = Pose_Para(5); t3dz = Pose_Para(6); f = Pose_Para(7); % scale

are different from the 8 data in the code？ mDim = idDim + 8

# Convert projection matrix into 4x3 matrices
m = tf.reshape(m, [-1,4,2])                                                                              # batch_size x 4 x 2

m_row1 = tf.nn.l2_normalize(m[:,0:3,0], axis = 1)
m_row2 = tf.nn.l2_normalize(m[:,0:3,1], axis = 1)
m_row3 = tf.pad(tf.cross(m_row1, m_row2), [[0,0],[0,1]], mode='CONSTANT', constant_values=0)
m_row3 = tf.expand_dims(m_row3, axis=2)

m = tf.concat([m, m_row3], axis = 2)                                                                      # batch_size x 4 x 3

[tranluan]

I actually estimate 2x4 projection matrix directly

[wangxinjun56]

so , i test a sample about the m ,i first output one ground true sample data: m ,s , image,texture, and mask. with the code: m = batch_m[0] self.std_m + self.mean_m np.savetxt("./cal/m.txt",m,fmt="%s",delimiter=",")
s = batch_shape[0] self.std_shape + self.mean_shape np.savetxt("./cal/s.txt",s,fmt="%s",delimiter=",")

                t2 = _300W_LP_DIR + 'image/'+ batch_300W_images_fn[0]
                t3 = _300W_LP_DIR + 'mask_img/'+ batch_300W_images_fn[0]
                t4 = _300W_LP_DIR + 'texture/'+ image2texture_fn(batch_300W_images_fn[0])
                t5 =  _300W_LP_DIR + 'mask/'+ image2texture_fn(batch_300W_images_fn[0])
                copyfile(t2, '/cal/1.png')
                copyfile(t3, '/cal/2.png')
                copyfile(t4, '/cal/3.png')
                copyfile(t5, '/cal/4.png')

and i use a code slice from your code ;

import numpy as np import tensorflow as tf def get_shape(tensor): dims = tensor.shape.as_list() return dims mshape = np.loadtxt('s.txt') m = np.loadtxt('m.txt') m = tf.reshape(m, [-1,4,2]) m_row1 = tf.nn.l2_normalize(m[:,0:3,0], axis = 1) m_row2 = tf.nn.l2_normalize(m[:,0:3,1], axis = 1) m_row3 = tf.pad(tf.cross(m_row1, m_row2), [[0,0],[0,1]], mode='CONSTANT', constant_values=0) m_row3 = tf.expand_dims(m_row3, axis=2) m = tf.concat([m, m_row3], axis = 2) n_size = 1 vertex3d = tf.reshape( mshape, shape = [n_size, -1, 3] ) # batch_size x vertex_num x 3 vertex4d = tf.concat(axis = 2, values = [vertex3d, tf.ones(get_shape(vertex3d)[0:2] +[1], tf.float64)]) # batch_size x vertex_num x 4

m = tf.cast(m, tf.float64) vertex2d = tf.matmul(m, vertex4d, True, True) # batch_size x 3 x vertex_num print("vertex2d shape:",vertex2d.shape) vertex2d = tf.transpose(vertex2d, perm=[0,2,1]) # batch_size x vertex_num x 2 print("vertex2d shape2:",vertex2d.shape) vertex2d_single = tf.split(axis = 0, num_or_size_splits = n_size, value = vertex2d) vertex2d_i = tf.squeeze(vertex2d_single[0], axis=0) s =224 [vertex2d_u, vertex2d_v, vertex2d_z] = tf.split(axis=1, num_or_size_splits=3, value=vertex2d_i) vertex2d_u = vertex2d_u - 1 vertex2d_v = s - vertex2d_v

vertex2d_i = tf.concat(axis=1, values=[vertex2d_v, vertex2d_u, vertex2d_z]) vertex2d_i = tf.transpose(vertex2d_i)

with tf.Session() as sess: out_v = sess.run(vertex2d_i) print("out_v.shape:",out_v.shape) outv = np.reshape(out_v,(-1,)) np.savetxt("rs.txt",outv,fmt="%s",delimiter=",")

finally ,i use the vertexs with the faceindexs, and found the result shape is strange;

the accuracy maxtrix : 0.0007021478967551096 -0.00010672902212836453 -2.7785656540952035e-05 0.000860147641784886 -0.0005394711305846331 -0.00018321523403917045 137.74349847449207 121.92588799400255

[tranluan]

If you use groundtruth shape and m, don’t multiply with mean and std.

[wangxinjun56]

when i don’t multiply with mean and std. the matrix : 0.07839612 -2.4697971 1.4430104 4.565652 -1.7359085 -0.09378817 1.6977928 -0.07866878

but the final result ,the final shape is also strange wrong;

[wangxinjun56]

[wangxinjun56]

and the shape basic is ok ,but after matrix ,the shape become bad .

[wangxinjun56]

so ,may be the m(matrix data) in *_param.dat have somewrong ? hope you double check, thanks.

[huyanfei-cqupt]

hi, could you tell me where are the m(matrix data) and shape datas come from ?

acording the paper says "With the face profiling technique, Zhu et al. [33] expand the 300W dataset [68] into 122, 450 images with fitted 3DMM shapes and projection parameters m."

but i can't find it, could you give me the accuracy network link?

hi， Can I ask you some questions? @wangxinjun56

[HOMGH]

and the shape basic is ok ,but after matrix ,the shape become bad .

Hi, I'm getting a (batch_size, 3*number of vertex) matrix as the estimated shape. Could you please clarify how can I use this to display the shape using meshlab? Do I need anything else other than this matrix?