search

Kinovarobotics / ros_kortex_vision

ROS package for KINOVA® KORTEX™ arms vision module
BSD 3-Clause "New" or "Revised" License
32 stars 34 forks source link

Align the color image frame with the depth image frame #16

Open MQ-D-729 opened 1 year ago

MQ-D-729 commented 1 year ago

Hello, is there an example update for the align color image and depth image topic `posted, similar to“/camera/aligh_depth/image" 480X270 and "/camera/align_color/image_raw" 480X270.

felixmaisonneuve commented 1 year ago

Hi @MQ-D-729,

You can look into the feature/calibration-program for a program and instructions that will allow you to align the depth sensor and color sensor images in moveIt.

It automates a bunch of stuff that will help you get the perfect alignement, but in the end, it just changes the parameters here : https://github.com/Kinovarobotics/ros_kortex_vision/blob/fb76366262c90cf735321ca81b005ee8171af487/launch/kinova_vision_rgbd.launch#L97

If you are not using MoveIt and you are trying to align the actual intel sensor, I know it is possible to disassemble the vision module to access and the USB-c port that can allow you to connect to the camera directly. You can then use the Intel software to configure you camera. I know a user who did something similar. I think it is possible to adjust the alignement from there, but I might be wrong since the depth sensor and the color sensor are not the same camera (I am not 100% sure about that part).

Let me know if this is not what you are looking for,

Best, Felix

Liuzhuoqiang commented 1 year ago

may i ask where to find the feature/calibration-program for a program and instructions

MQ-D-729 commented 1 year ago

Hi @MQ-D-729,

You can look into the feature/calibration-program for a program and instructions that will allow you to align the depth sensor and color sensor images in moveIt.

It automates a bunch of stuff that will help you get the perfect alignement, but in the end, it just changes the parameters here :

https://github.com/Kinovarobotics/ros_kortex_vision/blob/fb76366262c90cf735321ca81b005ee8171af487/launch/kinova_vision_rgbd.launch#L97

If you are not using MoveIt and you are trying to align the actual intel sensor, I know it is possible to disassemble the vision module to access and the USB-c port that can allow you to connect to the camera directly. You can then use the Intel software to configure you camera. I know a user who did something similar. I think it is possible to adjust the alignement from there, but I might be wrong since the depth sensor and the color sensor are not the same camera (I am not 100% sure about that part).

Let me know if this is not what you are looking for,

Best, Felix

This can indeed align the color image and the depth image at a fixed position, but it only changes the TF transformation of the depth camera relative to the color camera. This transformation should be fixed, and their relative positions remain unchanged. To get an aligned image, I think It needs the camera's distortion parameters, external parameters, internal parameters, and alignment through image algorithms, similar to pyrealsenses. It seems to be possible to use usb control after disassembly, but I am worried about disassembly. I am not very good at it, so it will be more troublesome. Is there any other way?

MQ-D-729 commented 1 year ago

may i ask where to find the feature/calibration-program for a program and instructions inside the branch，but,use ubnutu20.04

felixmaisonneuve commented 1 year ago

I do not know much about camera parameters. All the vision parameters that can be configured are detailed in the documentation. If these options are not enough, there is nothing you could do via ROS or the Kortex API.

If you decide to proceed to disassemble the vision module to access the camera directly, we can provide the instructions. You need to ask support@kinova.ca

Also, the ros_kortex_vision was not upgraded to ROS Noetic (ROS version of Ubuntu20.04), so you cannot currently use the repo and the calibration tool with Ubuntu20.04. I do not know how much work this would be, but if you decide to do it, you can look at this commit where I did the Melodic -> Noetic migration on our main repo.

Hope this helps a bit, Felix

MQ-D-729 commented 11 months ago

The official Intel D410 depth camera shows that it supports 1280x720 at 30fps, why is there only a 480X270 depth image in the launch file. And the color camera is still OV5640, not so easy to align depth frames。

felixmaisonneuve commented 11 months ago

The 1280x780 option is not supported by the API for the depth sensor. This is something that we are aware of and I bug has been open for this, but I do not have a timeline for the fix