Projecting 'j3d' onto image plane

[thohemp]

Hello,

thanks for your work. I try to project the 3D joints onto the image plane, but I don't seem to get the correct results.

First I define the camera intrinsics:

self.focal_length=600
fx,fy = self.focal_length, self.focal_length
height=512
width=512
self.camera_center = np.array([width / 2., height / 2.])

Then I take 3d joint of the index_fingertip:

index_fingertip = results['0'][0]['j3d'][7]

Translate it to camera coordinates:

index_fingertip+=results['0'][0]['cam_trans']

Finally, I calculate and print the pixel coordinates:

u = fx * (index_fingertip[0]/index_fingertip[2]) + self.camera_center[0]
v = fy * (index_fingertip[1]/index_fingertip[2]) + self.camera_center[1]
print("u: {} +  v: {}".format(u,v))

However, the results are not correct. Placing the finger tip on the left side of the image plan will output a u coordinate of 140. Similar results are reported for the y axis. What am I missing?

[ZhengdiYu]

Sorry for the late response due to my work here :) The 2D projection is done using the predicted weak-perspective camera parameters. And it's already in outputs['pj2d'].

To visualize, I suggest that you can add 'pj2d' into the show_items_list here: https://github.com/ZhengdiYu/Arbitrary-Hands-3D-Reconstruction/blob/3ece8dca875887430c82adf8c8cc887b79a01540/acr/main.py#L100

After that, the visualization of 2D keypoints will be done using the following code: https://github.com/ZhengdiYu/Arbitrary-Hands-3D-Reconstruction/blob/3ece8dca875887430c82adf8c8cc887b79a01540/acr/visualization.py#L234-L244