Closed Camilochiang closed 2 years ago
Hi @Camilochiang It may be useful to compare your Python decompression code to a decompression Python node script written by another RealSense ROS user in https://github.com/IntelRealSense/realsense-ros/issues/1512 that utilizes the /camera/depth/image_rect_raw/compressed topic.
Like yourself, that script is using the bridge.compressed_imgmsg_to_cv2() instruction but additionally defining bgr8 in the brackets of the instruction.
Hei @MartyG-RealSense ! thanks for your answer but that doesn't solve the problem :). I guess that is either a bug or my faul ..
I manage to get the colorized (8 bits as is explained in README.me), soto keep going I'm following the tutorial that you have mention in a several times but just curious : is it possible to configure is_disparity
from a ROS launch file? Does that variable exist in the actual software?
I manage already to change the laser power and turn off histogram_equalization_enabled
with the following code, but i don't get the inverse colorization as i would expect....
<rosparam>
/camera/stereo_module/laser_power: 300
/camera/colorizer/histogram_equalization_enabled: false
/camera/colorizer/is_disparity: true
</rosparam>
Thanks!
If you are using Python then I would recommend not using the depth compression by colorization white-paper because of the difficulties involved in performing the depth recovery process outside of the RealSense SDK and in creating a Python conversion of the paper's recovery logic.
There is a long and detailed discussion with scripting about Python compression and decompression in https://github.com/IntelRealSense/librealsense/issues/8117 that might provide helpful insights.
It looks as though in the colorization paper, is_disparity is simply a bool for switching between applying depth to disparity or disparity to depth. One should be true and one false. Both should not be false or true. A Python script in the link below provides an example of setting True and False statuses for the two with Python code.
It is possible to define a disparity filter in a ROS launch. It could be included in the roslaunch instruction with the 'disparity' filter definition, like this:
roslaunch realsense2_camera rs_camera.launch filters:=disparity
https://github.com/IntelRealSense/realsense-ros/issues/501 provides an example of defining filters in a launch file.
To get a list of available parameters for the disparity filter, please try running:
rosrun dynamic_reconfigure dynparam get /camera/disparity
Hei @MartyG-RealSense once again for your help. I would comment some problems related to the mentioned:
/camera/depth/image_rect_raw/compressed
give a 8 bit picture instead of 16 bit. Any way to check it up? Also what is the topic /camera/depth/image_rect_raw/compressedDepth
publishing? Is also 8 bits....In another topic,, as I already have your attention: What is the function of enable_infra1
and enable_infra2
? I could not find any documentation about it. All what I found was the note about infra_rgb
on README.md
Thanks!
enable_infra1 and enable_infra2 enable the publishing of the topics for the left and right infrared streams respectively, which are disabled by default in the RealSense ROS wrapper.
In regard to your compression question, I will refer that to Doronhi the RealSense ROS wrapper developer. Thanks very much for your patience.
Hi, Regarding the decompression of 16bit image, I managed to reproduce without the rosbag. While listening on "/camera/depth/image_rect_raw/compressed" and getting the message into "res":
bridge = CvBridge()
depth_image = bridge.compressed_imgmsg_to_cv2(res)
res.format %yields: '16UC1; jpeg compressed '
depth_image.dtype %yields: dtype('uint8')
I think this error is already dealt with here It is not strictly related to realsense-ros and although the PR was merged 20 days ago I don't know when the fix will be released.
Hei @doronhi Thanks! I just must mention that /camera/color/image_raw/compressed
is also giving me a 8 bit variable
>>> for topic, msg, t in bag.read_messages():
... print(topic)
/camera/color/image_raw/compressed
>>> dir(msg)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '_check_types', '_connection_header', '_full_text', '_get_types', '_has_header', '_md5sum', '_slot_types', '_spec', '_type', 'data', 'deserialize', 'deserialize_numpy', 'format', 'header', 'serialize', 'serialize_numpy']
>>> msg.format
'rgb8; jpeg compressed bgr8'
Any information regarding about this issue?
The color images are 3 channel 8-bit images. What is unexpected with 'rgb8; jpeg compressed bgr8' for topic '/camera/color/image_raw/compressed'? Am I missing something?
Hei @doronhi . Please correct me if I'm wrong. The official Datasheet for the Intel Realsense product family D400 series (Revion 009 - June 2020) says that the picture should be YUY2 16 bits for the color Stream of the RGB camera (in my case 415). Isn't that the case? My apologies if is me who is confuse .
No need to apologize. The camera's format is converted to RGB8 using librealsense2 functions to follow the ros2 standard sensor_msgs::image_encodings::RGB8. We could make realsense-ros send the color topic in other formats but I don't remember that the need ever came up.
ah, OK. I don't think that is the case, as ROS is designed for robotic applications and ML have demonstrated so far to work better in 8 bits that 16 bits. Thanks for your help!
@MartyG-RealSense I actually realize now, looking to some files that you never answer one question. What is the topic compressedDepth publishing it? Thanks!
Hi @Camilochiang I emphasize upfront that I am not certain about this particular question. My research suggests to me that compressedDepth is publishing depth data in a compressed state through compressed_depth_image_transport, as described at https://github.com/IntelRealSense/realsense-ros/issues/369 and https://github.com/IntelRealSense/realsense-ros/issues/1672
Thanks @MartyG-RealSense ! Not 100% sure about the difference between compress and compressDepth topic but i will investigate it
in the other hand @doronhi, I did modify manualy cv_bridge and now is working (so python return 16bit from compressed msg). I confirm using the code from the PR. But still, when I decompress using
depth_image = CvBridge().compressed_imgmsg_to_cv2(msg)
i get 8 bit values
and using
depth_image = CvBridge().compressed_imgmsg_to_cv2(msg, desired_encoding="bgr16")
I get wrong values (like 65 m, meanwhiles my max distance was 7m). Could you have a look to see what encoding should be use? With this last options all values are either close to 65m or 0, giving me a "monochromatic" image.
``
@Camilochiang Somebody at https://github.com/ros-perception/vision_opencv/issues/206 experienced the same issue with compressed_imgmsg_to_cv2(msg) returning an 8-bit image.
Hi @Camilochiang Do you require further assistance with this case, please? Thanks!
Hei @MartyG-RealSense yes :) Im still waiting for the answer of @doronhi . Im just not sure if the message is really arriving as an 8 bit or I dont know how to decompress it ;)
Would using BRG8 for desired_encoding instead of BGR16 resolve your problem, please?
depth_image = CvBridge().compressed_imgmsg_to_cv2(msg, desired_encoding="bgr8")
@MartyG-RealSense , I did try with all encoding (bgr8, bgr16, passthrough) but after decompressing the image I only get values really high values (65k) or 0 as I told you before. I will retry on Monday and let you know. Thanks for your help!
You are very welcome. I will be happy to continue providing support in your case. Doronhi is no longer working with RealSense ROS and the focus of RealSense ROS wrapper development is now on the ROS2 4.x wrapper, with further updates not planned for the ROS1 wrapper.
Hi @Camilochiang Do you require further assistance with this case, please? Thanks!
I keep working on it. To avoid the usage of cv_bridge I tried with imageio (To read as bytes), but the compressed ros message keep coming out as a 8bits value, what make useless compresion in depth images.
image = io.BytesIO(last_msg.data)
iio.imread(image)
Array([[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0],
[ 0, 0, 0, ..., 0, 0, 0],
...,
[ 0, 0, 0, ..., 255, 255, 0],
[ 0, 0, 0, ..., 247, 254, 0],
[ 0, 0, 0, ..., 0, 0, 5]], dtype=uint8)
# Lets corroborate it that 16 bits works with imageio
im1 = np.zeros((3,3), np.uint16)
im1[0,0] = 1000
iio.imsave('test.png', im1)
im2 = iio.imread('test.png')
print(im1.dtype, im1[0, 0]) # -> uint16 1000
uint16 1000
print(im2.dtype, im2[0, 0]) # -> uint16 1000
uint16 1000
I conducted further research on the subject. The official RealSense SDK ROS documentation link below illustrates the difference in published topics when using compressed-image-transport and compressed-image-depth-transport
https://dev.intelrealsense.com/docs/ros-wrapper#section-compression-packages
I also located a past RealSense case with a similar situation to your own, where someone was attempting to obtain a 16UC1 image from compression and getting an 8-bit 8UC1 image instead.
The above discussed linked to this one:
https://answers.ros.org/question/276352/how-to-understand-data-from-the-image_rawcompressed-topic/
Hi @Camilochiang Do you require further assistance with this case, please? Thanks!
Hei @MartyG-RealSense . I tried another two approach and either I dont know how to recover the images as 16 bits or there is a bug
>>> depth.format
'16UC1; jpeg compressed '
>>> depth = np.frombuffer(depth.data, dtype=np.uint16).reshape(720, 1280, -1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: cannot reshape array of size 114409 into shape (720,1280,newaxis)
>>> depth = np.frombuffer(depth.data, dtype=np.uint8).reshape(720, 1280, -1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: cannot reshape array of size 228818 into shape (720,1280,newaxis)
>>> depth = io.BytesIO(depth.data)
# in terminal: rosrun image_transport republish compressed in:=/camera/depth/image_rect_raw out:=/test
# After starting the subscriber...
>>> np.frombuffer(test.msg.data, dtype=np.uint16).reshape(720, 1280)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: cannot reshape array of size 460800 into shape (720,1280)
>>> np.frombuffer(test.msg.data, dtype=np.uint8).reshape(720, 1280)
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]], dtype=uint8)
In both cases only 8 bit are working. Is there any way to check if this is a bug or not? Greetings
I do not have any further advice that I can offer on this subject, unfortunately. If there was a bug in the ROS wrapper then there would not be a way to resolve it as the RealSense ROS wrapper development team are now focusing on the ROS2 4.x series of wrappers and the ROS1 2.x wrapper series will not receive further updates.
Ok. Closed without a satisfactory answer. Thanks in any case!
I do not have any further advice that I can offer on this subject, unfortunately. If there was a bug in the ROS wrapper then there would not be a way to resolve it as the RealSense ROS wrapper development team are now focusing on the ROS2 4.x series of wrappers and the ROS1 2.x wrapper series will not receive further updates.
Hi,I also met the issue when I was using the ros2 wrapper for getting the compressed depth frame(D435, Foxy, cv_bridge, Python). Is there any better way to solve that ?
Hi @Zhuomin-Zhou At the bottom of the page linked to below - in the section titled Image transport problems and compressed topics - it is reported that there have been problems for ROS2 Foxy users in general (not the RealSense ROS2 wrapper specifically) with the ROS2 image_transport_plugins package, with links provided to example cases about the problem.
https://www.stereolabs.com/docs/ros2/
In the links that the above page recommends, a ROS2 Foxy user suggested a method for fixing the issue at https://github.com/stereolabs/zed-ros2-wrapper/issues/31#issuecomment-953447841
Hi @Zhuomin-Zhou At the bottom of the page linked to below - in the section titled Image transport problems and compressed topics - it is reported that there have been problems for ROS2 Foxy users in general (not the RealSense ROS2 wrapper specifically) with the ROS2 image_transport_plugins package, with links provided to example cases about the problem.
https://www.stereolabs.com/docs/ros2/
In the links that the above page recommends, a ROS2 Foxy user suggested a method for fixing the issue at https://github.com/stereolabs/zed-ros2-wrapper/issues/31#issuecomment-953447841
Thank you for your help!!
Platform : Rpi4 + Ubuntu 18.04 Camera : D415 rosversion: 1.14.11 rosdistro: melodic rosversion realsense2_camera : 2.3.1
After recording a rosbag, the topic
/camera/depth/image_rect_raw/compressed
give only values up to 256 when using the following code for decompress in python but this is not the case with uncompressed images (with its normal 16 bits)All the examples that i found were done with uncompressed data and the only issue that works with compressed data, is also working with 8 bits