how to convert 2.5D coordinate to real-world 3D coordinate?

[Joyako]

Hi, thanks for your excellent project! Q1: What coordinate system is the output of the model directly? Q2: I found in your paper 4.4 how the 3D coordinates of the hand are calculated as fllows:

but I can not understand that what the meaning of camera back-projection and inverse affine transformation and how to calculate it in your code. Looking forward to your reply, thanks.

[mks0601]

Q1. x,y: pixel (0~63, 0~63), z: depth (normalized to 0~64) Q2. camera back-projection: (x_img, y_img, z_real) -> (x_real, y_real, z_real). inverse affine transformation: cropped and resized hand image space -> original image space before cropping and resizing

[Joyako]

@mks0601 thanks a lot, camera back-projection: (x_img, y_img, z_real) -> (x_real, y_real, z_real) Assuming that the camera internal parameters are know , it can be solved by the following formula, right? z_real = z_real x_real = (x_img - cx) z_real / fx y_real = (y_img - cy) z_real / fy

camera internal parameters: K = [[fx, 0, cx], [0, fy, cy], [0, 0, 1]]

[mks0601]

yes. that function is implemented in utils.transformations.pixel2cam

[Joyako]

thanks, I will close it.