3D projection problem

[eldentse]

Hi,

Thank you so much for this work! I am currently having issues on projecting object 3D points (from mesh) back to image plane using both camera intrinsics and extrinsics. Would you mind having a quick look on the following code please?

`

    label = np.load(sample['label_file'])
    trans = label['pose_y'][sample['ycb_grasp_ind'],:,:].astype(np.float32)
    trans_hom = np.concatenate((trans,np.array([[0,0,0,1]],dtype=np.float32)),axis=0).transpose()

    cam_extr = np.array(sample['cam_extr']).astype(np.float32).reshape(3,4) #I have added this to original code and double-checked
    cam_extr_hom = np.concatenate((cam_extr,np.array([[0,0,0,1]],dtype=np.float32)),axis=0)

    verts = obj["verts"]
    hom_verts = np.concatenate([verts, np.ones([verts.shape[0], 1])], axis=1)
    verts_w = trans.dot(hom_verts.T).T
    vert_cam = cam_extr.dot(trans_verts.transpose()).transpose()[:, :3]

    #Here I have skipped how I obtained intr for clarity
    cam_intr = np.array([[intr['fx'], 0.0, intr['ppx']],
                                     [0.0, intr['fy'],intr['ppy']],  
                                     [0.0, 0.0, 1.0]])
    hom_2d = (np.array(cam_intr).dot(objpoints3d.transpose()).transpose())
    objpoints2d = (hom_2d / hom_2d[:, 2:])[:, :2]

`

[ychao-nvidia]

First, label['pose_y'] already gives you object pose in the camera frame so you don't need any extrinsics (i.e., cam_extr, cam_extr_hom) here.

Also, I don't think you need .transpose() for trans_hom.

[eldentse]

Thank you so much for the quick reply!