how tracked3D_normalized_pts_fix_contour transforms to tracked2D_normalized_pts_fix_contour using given info?

[foocker]

in your paper, camera is at (0, 0, 0 ) and no rotation, so only camera intrinsic can do 3d -> 2d. but i have test the given file which in inference, i got the wrong result. my questuon is: camera_intrinsic,+change_paras, do tracked3D_normalized_pts_fix_contour->tracked2D_normalized_pts_fix_contour . how?

··· pts_3d = tracked3D_normalized_pts_fix_contour[0, 2, :] td = camera_intrinsic.dot((pts_3d+1)/2) td = td[:2] /td[-1] ···

[YuanxunLu]

Camera parameters project 3d points to 2d image coords. You can draw the projected 2d landmarks to check whether you are right.

[foocker]

Camera parameters project 3d points to 2d image coords. You can draw the projected 2d landmarks to check whether you are right. ··· k = 5 # 数据中第k个 camera = Camera() import scipy.io as sio from os.path import join data_root = './data/Obama2' scale = sio.loadmat(join(data_root, 'id_scale.mat'))['scale'][0,0] # 来自3D人脸追踪, 一个数

pred_headpose = np.zeros((10, 6), dtype=np.float32) # 这里测试3D->2D，没有旋转，没有平移

fit_data = np.load('./data/Obama2/3d_fit_data.npz') trans = fit_data['trans'][:,:,0].astype(np.float32) # (13369, 3) rot_angles = fit_data['rot_angles']

pred_headpose[k, 3:] = trans[k] pred_headpose[k, 0] += 180

final_pts3d = tracked3D_normalized_pts_fix_contour

predlandmarks, , _ = project_landmarks(camera_intrinsic, camera.relative_rotation, camera.relative_translation, scale, pred_headpose[k], final_pts3d[k]) print(pred_landmarks[k], 'gg', tracked2D_normalized_pts_fix_contour[k][k]) ··· from above code i can (equal at a small error), translate 3D(tracked3D_normalized_pts_fix_contour) to 2D(tracked2D_normalized_pts_fix_contour). but there are still a small gap. [and shoulder can also tranlate by the same way]

my question is:

1. what's the difference of pts_3d = fit_data['pts_3d'] and tracked3D_normalized_pts_fix_contour, （from tracked3D_normalized_pts_fix_contour.npy).
2. pts_3d = fit_data['pts_3d'] is from the 3D face traking (like DECA), and the tracked3D_normalized_pts_fix_contour is handle using what func to get the result?
3. ..in utils.project_landmarks(camera_intrinsic, camera.relative_rotation, camera.relative_translation, scale, pred_headpose[k], final_pts3d[k]) scale =f/dx? pred_headpose for headpose trans, rotation, what's the coordinate? final_pts3d only contain mouth's 46：64 landmarks and eye_brow landamarks, on the fix_countour coordinate. i think, i understand the camer model, but here still some confusion,such as :pts3d_headpose = scale * rot.dot(pts_3d.T) + trans # 相机物理？or what's it means 👍 about utils.project_landmarks function? @YuanxunLu