This research project implements a real-time object detection and pose estimation method as described in the paper, Tekin et al. "Real-Time Seamless Single Shot 6D Object Pose Prediction", CVPR 2018. (https://arxiv.org/abs/1711.08848).
MIT License
725
stars
214
forks
source link
100% accuracy but incorrect transformation matrix #91
So I don't think there is a problem with the model detecting the object but the location of the object being detected with respect to the camera coordinates. My end goal is to use singleshotpose in a robot environment.
I am placing the camera (in unreal engine) at [0,0,0,] such that my camera coordinates correspond to the world coordinates.
My object is stanford bunny in which its translation vector is [13,10,35] in the image. According to this coordinate system, as OpenCV expects, X axis points to the camera’s right, Y axis down and the Z axis towards the scene.
According to the paper:
In our case, PnP uses only 9 such control point correspondences and provides an estimate of the 3D rotation R and 3D translation t of the object in camera coordinates.
So if I am understanding it right, I shall be able to infer the 6D pose of the object w.r.t the camera coordinates (which is the world coordinates as well in my case).
For simplicity, I am feeding 150 images for the same pose of the object.
The model's translation vector ground truth is [0.1071573, 0.09952634, 0.35997307] and the predicted result is [0.1040645, 0.10019193, 0.36093822]. Same issue with the rotation matrices.
I suspect its a configuration issue and would be glad to receive any help. Please let me know for any more details. Looking forward for your reply.
Hello everyone,
So I don't think there is a problem with the model detecting the object but the location of the object being detected with respect to the camera coordinates. My end goal is to use singleshotpose in a robot environment.
I am placing the camera (in unreal engine) at [0,0,0,] such that my camera coordinates correspond to the world coordinates.
My object is stanford bunny in which its translation vector is [13,10,35] in the image. According to this coordinate system, as OpenCV expects, X axis points to the camera’s right, Y axis down and the Z axis towards the scene.
According to the paper: In our case, PnP uses only 9 such control point correspondences and provides an estimate of the 3D rotation R and 3D translation t of the object in camera coordinates. So if I am understanding it right, I shall be able to infer the 6D pose of the object w.r.t the camera coordinates (which is the world coordinates as well in my case). For simplicity, I am feeding 150 images for the same pose of the object.
I tried solve.pnp and solve.ransac and both gave me close results. As for PnP arguments,my bouding box is correct as you can see above, my object dimensions PLY is correctly scaled and the camera intrinsics are well defined. https://github.com/microsoft/singleshotpose/blob/766d5eced25df6f0c438c6414c07b8b686c0558e/train.py#L206-L208
The model's translation vector ground truth is [0.1071573, 0.09952634, 0.35997307] and the predicted result is [0.1040645, 0.10019193, 0.36093822]. Same issue with the rotation matrices.
I suspect its a configuration issue and would be glad to receive any help. Please let me know for any more details. Looking forward for your reply.