How to get distance information from realsense attached to the Gen3 arm

[gustn6591]

I succeeded in forming the depth map via cv2.CAP_GSTREAMER. But i don't know How to get distance in this depth map. This is because the information in the depth map is represented by integers. Please tell me how to get distance information from the acquired depth map.

Below is the code I wrote in window10, python3.7

cap_depth = cv2.VideoCapture("rtsp://admin:admin@192.168.1.10/depth", cv2.CAP_GSTREAMER) while(cap_depth.isOpened()): ret2, frame_depth = cap_depth.read() cv2.namedWindow('kinova_depth', cv2.WINDOW_AUTOSIZE) cv2.imshow('kinova_depth',frame_depth) if cv2.waitKey(20) & 0xFF == ord('q'): break cap_depth.release() cv2.destroyAllWindows()

[felixmaisonneuve]

Hi @gustn6591,

This is interesting that you were able to make it work using openCV, I thought it was not possible because the depth sensor uses a 16-bit raw stream: https://github.com/Kinovarobotics/kortex/issues/49#issuecomment-635425077

I am not very familiar with the vision module, what values do you get? I know you can set a min and max value for the camera range, I would assume every value correspond to a ratio within that range.

To use the vision module, we usually recommand to use the ros_kortex_vision repo. We also have the feature/calibration-program branch with a small program that help play with the depth sensor

Best, Felix

[gustn6591]

Sorry, What I got is not a depth map, just a 1D gray scale image

As far as I know, I know that color or depth images are obtained through rtsp from the kinova model. I would like to know an example or pipeline for this method available on Windows 10.

[felixmaisonneuve]

Our best examples can be found in the ros_kortex_vision repo which is only accessible with Ubuntu.

On windows, you are limited to what is detailed in the User guide in the "Accessing Vision module color and depth streams". Essentially, this is accessing camera sensors via the rtsp://<base IPv4 address>/depth and rtsp://<base IPv4 address>/color streams

[gustn6591]

Is the only way using opencv to check images via rtsp:///depth?

When checking the depth image through opencv, the pixel data type was 8bit. The information I want to get is 16bit of the depth image. Is there any other way to check the image?

Even if you click Python vision examples to find the corresponding example on page 84 of the user manual, you will only receive an answer that the page cannot be found.

[felixmaisonneuve]

You can access the depth sensor using Gstreamer. This is what we do in the ros_kortex_vision repo. This is what I would recommand to use.

I think you are using an older version of the user guide (rev6) and you actually refer to the link on page 74. The example folder was renamed from "500-Vision" to "500-Gen3_vision_configuration" and this is why the link is not working

Here is the correct link for the vision examples.

These examples show how to configure the vision module and not how to manipulate streams, so I do not think this is what you are looking for.