Doesn't handle camera rotations proplery

[kingsman142]

I'm currently using the Realsense D415 camera to gather RGBD data of an object, specifically a bottle. This camera is attached to the end-effector of a 6DOF robot. I use the robot to gather real-world pose coordinates of the camera. I use this kinematics data as the pose of the camera for both translation and rotation. This program works if I perform translation across one axis. However, it doesn't work when I perform a rotation.

Reading the papers linked in the README, I can't seem to figure out if their pose data is relative to the world's origin or if they're absolute physical locations, such as ones received from a robot's kinematics data. Do you know if I should perform any additional transformations on my pose data, similar to how the C++ programs calculate the inverse of the base frame and multiply that with each subsequent frame?

The first image is with the camera just performing a translation across one axis. The second image is the camera performing a translation across the same axis, but also performing a 0 to 360 degree rotation along the way.

EDIT: By the way, I have double-checked the camera intrinsics, and have used two different realsense depth cameras and I'm encountering similar issues in both. Also, I have verified the positions of the end-effector (which is carrying the camera) are correct.

[jly0810]

Hello, I have encountered a similar problem. How did you solve it? This problem has troubled me for a long time. Thank you

[H-tr]

This might because of different world coordination type. This project by default is using OpenCV camera coordinate system while your input might be OpenGL coordinate system. To convert it, you can try

mat[2, :] *= -1
mat = mat[np.array([1, 0, 2, 3]), :]
mat[0:3, 1:3] *= -1

where mat is your extrinsic.