NoneJou072 / robopal

robopal: a multi-platform, modular robot simulation framework based on MuJoCo, mainly used for reinforcement learning and control algorithm implementation of robotic arms.
https://robopal.readthedocs.io/
Apache License 2.0
135 stars 14 forks source link

about camera's intrinsics #29

Open kunpeng-wu opened 1 month ago

kunpeng-wu commented 1 month ago

I want to use the camera's intrinsics to convert pixel coordinates to camera coordinates, and then use the extrinsics to convert the camera coordinates to world coordinates. If I use the intrinsics in cv_utils.py, I didn't get the correct result. But by setting the fy to -fy, I get the correct result, can you tell me why is that? my camera is

<camera name="0_cam" pos="0.5 0 1" quat="0.707107 0 0 -0.707107"/>

the code I used as follows

K = get_cam_intrinsic()
K_inv = np.linalg.inv(K)
T = np.array([[0, 1, 0, 0.5], [-1, 0, 0, 0], [0, 0, 1, 0.8], [0, 0, 0, 1]])  # extrinsics

pixel = np.array([center_x, center_y, 1, 1]).reshape(4, 1)  # pixel coordinates
p_c = K_inv @ pixel
p_w = T @ p_c

Deepmind Control Camera Matrix

In deepmind's code, they negate the fx, but in my case, fy should be negated.