Closed Steven-ZY closed 4 years ago
MartyG-RealSense
commented
4 years ago @Steven-ZY Although the D435i and the RealSense T265 Tracking camera use the same IMU component, they process the data in different ways. So the T265 has built in support for accessing 6DOF pose data but the D435i does not.
There are workarounds though to get the pose of objects with a D435 type camera using the vision softwares OpenCV or ROS.
Steven-ZY
commented
4 years ago @MartyG-RealSense Sir，thanks a lot for your reply. After reading what you introduced to me, I still have two questions. Firstly, my goal is to find the 3D pose of an object in the real world with regard to the camera coordinate using the D435i, not acquire the pose of the camera. And note that I need to use the depth data, so maybe solvePnP in OpenCV is not completely suitable for me. In a word, how can I **acquire the pose of an object** using **RGB and depth** data in the D435i? Secondly, I noticed that you once mentioned the dynamic calibration tool , what's the difference between it and the calibration at the factory? the calibration at the factory only include the calibration about the RGB camera? how about the two Infrared projectors? And where/how to find camera calibration matrix, distortion matrix and maybe others for real sense D435i ? I'm looking forward to your reply, thanks in advance.
MartyG-RealSense
commented
4 years ago @Steven-ZY I will follow up with pose in another comment after this one.
All imagers on the RealSense 400 Series cameras are calibrated at the factory, and so calibration is usually very good when the camera is new. There are events though that can cause the camera to become mis-calibrated, negatively affecting the output image. These include a physical shock such as a hard knock or drop on the floor, severe vibration, temperature, or (in the case of the caseless Depth Modules), flexing the Depth Module board when handling it. A 'Dynamic Calibration' can greatly improve image quality in the most severe cases of disruption.
A factory calibration may calibrate both the intrinsics and extrinsics of the camera. The free public Dynamic Calibrator tool only calibrates the extrinsics, as it is these parameters that have the greatest effect on image quality.
It is possible to obtain the OEM version of the Dynamic Calibrator software (for calibrating both extrinsics and intrinsics) by purchasing the $1500 USD OEM Calibration Target board from the RealSense Store. The price is due mostly to the cost of manufacturing the large physical calibration board, and it is a product aimed at engineering departments and manufacturing facilities.
https://store.intelrealsense.com/buy-intel-realsense-d400-cameras-calibration-target.html
Customers who have a Non Disclosure Agreement (NDA) with Intel may alternatively be able to obtain the OEM calibrator software by enquiring through their special Zendesk support channel to their local Intel representative.
There is also a lightweight quick and easy calibration mode called "On-Chip Calibration" that is now available in the RealSense Viewer. This is not as advanced a calibration as the one that the Dynamic Calibrator tool can provide but is useful for quick adjustments of the depth image noise.
In regard to accessing the details of the calibration information of a camera, the SDK's rs-enumerate-devices tool has a "-c" mode that can be used to list the calibration details.
https://github.com/IntelRealSense/librealsense/tree/master/tools/enumerate-devices
MartyG-RealSense
commented
4 years ago In regard to other ways of detecting the pose of objects with a 400 Series camera: you could attach "aruco" image tags to the objects to enable the camera to find their pose. The links below have examples for doing so with RealSense:
https://github.com/Roboy/roboy_realsense
https://arxiv.org/pdf/1804.01708.pdf
In cases where you want to be able to have a camera identify - without use of a tag - not only the pose of an object but also whether it is a desired type of object (e.g a ripe strawberry or a certain type of component) so that a robot arm can select, pick and handle it, these typically use some kind of additional AI training solution. Here is an example case:
https://support.intelrealsense.com/hc/en-us/community/posts/360043117214-Need-HELP-
LIPS Corp also have a pre-made RealSense powered robot arm solution called LIPSArm:
Steven-ZY
commented
4 years ago @MartyG-RealSense Dear Sir, Thank you for your quick reply and guidance. I will go through the link and will get back to you if found any query.
Steven-ZY
commented
4 years ago @MartyG-RealSense Thank you. But are there any code examples about finding the pose of an object in the real world using D435 or D435i? I mean the pose of an object by combining the RGB data with depth data, not to get the pose of the camera.
MartyG-RealSense
commented
4 years ago The 'roboy' Aruco tag example should definitely meet your criteria of tracking object pose instead of the camera pose, as it is tracking the pose of the image tags attached to the objects.
https://github.com/Roboy/roboy_realsense
There is another Aruco tag pose project here:
https://github.com/njanirudh/Aruco_Tracker
I could not find any that were specific to D435 / D435i though, and pose estimation can become a very complicated subject. I could not find any tag-less tracking methods that were simple enough to be just involve tracking object pose with RGB and depth, and not require other additional systems such as AI.
MartyG-RealSense
commented
4 years ago Case closed due to no further comments received.
Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):
All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)
Issue Description
<Describe your issue / question / feature request / etc..>