could not convert image from '16SC1' to 'rgb8' ([16SC1] is not a color format. but [rgb8] is. The conversion does not make sense) ROS

[aristow1]

Hi, i need a small help with some matter that i cant wrap my head arround. it is not considering these repository but i may find some help here and that would appreciated. Im trying to publish a disparity map of a stereo camera, i wrote this node here and i keep getting this error when after starting the node : could not convert image from '16SC1' to 'rgb8' ([16SC1] is not a color format. but [rgb8] is. The conversion does not make sense) this is my node :

import rospy
import cv2 as cv
import numpy as np
from cv_bridge import CvBridge, CvBridgeError
from sensor_msgs.msg import Image
import message_filters

def read_cameras():
    imageL = message_filters.Subscriber("/camera_left/image_raw", Image)
    imageR = message_filters.Subscriber("/camera_Right/image_raw", Image)

    # Synchronize images
    ts = message_filters.ApproximateTimeSynchronizer([imageL, imageR], queue_size=10, slop=0.5)
    ts.registerCallback(image_callback)
    rospy.spin()

def image_callback(imageL, imageR):
    br = CvBridge()
    rospy.loginfo("receiving Image")

    imageLeft = br.imgmsg_to_cv2(imageL, "rgb8")
    imageRight = br.imgmsg_to_cv2(imageR, "rgb8")

    imageL_new=cv.cvtColor(imageLeft, cv.COLOR_BGR2GRAY)

    imageR_new=cv.cvtColor(imageRight, cv.COLOR_BGR2GRAY)

    stereo = cv.StereoBM_create(numDisparities=16, blockSize=15)
    disparity = stereo.compute(imageR_new,imageL_new)

    rospy.loginfo("showing depth image")
    # Process images...

    pub = rospy.Publisher('/left_camera', Image, queue_size=1)
    cv.imshow('disparity', disparity)
    msg = br.cv2_to_imgmsg(disparity, encoding='16SC1')
    pub2 = rospy.Publisher('/Right_camera', Image, queue_size=1)
    msg2 = br.cv2_to_imgmsg(disparity, encoding='16SC1')

    pub.publish(msg)
    pub2.publish(msg2)

if __name__ == '__main__':
    rospy.init_node('my_node')
    try:
        read_cameras()

    except rospy.ROSInterruptException:
        pass

is this an encoding Problem ? can someone tell me what im doing wrong exactly? i tried multiple approaches as to change the encodings but that didnt change. Anyhelp would be appreciated. thanks

[ijnek]

The issue is with converting a grayscale image into rgb8 without specifying how you want to convert it, but it looks like you try and convert your grayscale image unnecessarily to a color format. Rather than converting your binary image into rgb8, then immediately converting it back to gray as shown here:

imageLeft = br.imgmsg_to_cv2(imageL, "rgb8")
imageRight = br.imgmsg_to_cv2(imageR, "rgb8")
imageL_new=cv.cvtColor(imageLeft, cv.COLOR_BGR2GRAY)
imageR_new=cv.cvtColor(imageRight, cv.COLOR_BGR2GRAY)

try reading it in as a mono8 or another grayscale format instead, like following:

imageL_new = br.imgmsg_to_cv2(imageL, desired_encoding='mono16')
imageR_new = br.imgmsg_to_cv2(imageR, desired_encoding='mono16')

[aristow1]

@ijnek Thank you for your response. however after trying your solution im now getting this error :

disparity = stereo.compute(imageR_new,imageL_new)
error: /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/calib3d/src/stereobm.cpp:1072: error: (-210) Both input images must have CV_8UC1 in function compute

[ijnek]

Hi, try replacing 'mono16' with 'mono8' in the commands I sent previously and see if it works.

[aristow1]

Hi, try replacing 'mono16' with 'mono8' in the commands I sent previously and see if it works.

@ijnek i made the changes but i still cant see the disparity image in the desired topic with rqt_image_view and i get this this in the terminal : ImageView.callback_image() could not convert image from '16SC1' to 'rgb8' ([16SC1] is not a color format. but [rgb8] is. The conversion does not make sense)

FYI : im basically trying to estimate a disparity map from the subscribed image topics and then publish it. i set in both cameras launch file the color mode to : rgb8 Update : I changed the camera parameter in the launch file to mono8 but that didnt change anything, but it was worth the try but i still am confused on how to solve the issue

[ijnek]

I believe the problem lies in the input image format being the wrong format.

You seem to be expecting both input images to be color images. In that case, the code you have in the original post seems correct. The error messages indicate however, that the input images you're receiving are 16SC1 which are grayscale (or depth) images, not colored images.

Did you try using rqt_image_view to view the input images to see if they're gray or rgb?

[aristow1]

@ijnek

in ImageView i dont see anything when i select either the /left_camera or /right_camera topics and thats when i get that error in the terminal. however my /image_raw topic is rgb

[ijnek]

I believe your input images aren't rgb. What does /camera_left/image_raw and /camera_right/image_raw look like in rqt_image_view?

[aristow1]

@ijnek here are 2 screenshots of both /camera_left/image_raw and /camera_right/image_raw and a screenshot of the color mode im running the cameras with : rgb8

[ijnek]

Those images look like color images, and so they're definitely not gray. Could you try something along the lines of

print(imageL.encoding)
print(imageR.encoding)

to make sure that's rgb8?

[aristow1]

@ijnek

[INFO] [1668765614.833292]: receiving Image
[INFO] [1668765614.879525]: showing depth image
rgb8

yeah its rgb8 encoding :/

[ijnek]

Then I don't know why you're getting the error message could not convert image from '16SC1' to 'rgb8' ([16SC1] is not a color format. but [rgb8] is. The conversion does not make sense).... you'll have to narrow down which line is the problem.

But I did realise you have cv.COLOR_BGR2GRAY, but it should be cv.COLOR_RGB2GRAY..

[aristow1]

@ijnek thank you for your help. i noticed that as well and i changed it tocv.COLOR_RGB2GRAY but that didn't solve the Problem. what i get out of the disparity = stereo.compute(imageR_new,imageL_new) function has a dtype of np.int16. but does that have a role in the issue ?

[aristow1]

@ijnek i made the following changes to my code :

#! /usr/bin/env python2
import rospy
import cv2 as cv
import numpy as np
from cv_bridge import CvBridge, CvBridgeError
from sensor_msgs.msg import Image
import message_filters

def read_cameras():
    imageL = message_filters.Subscriber("/ids_left/image_rect", Image)
    imageR = message_filters.Subscriber("/ids_middle/image_rect", Image)

    # Synchronize images
    ts = message_filters.ApproximateTimeSynchronizer([imageL, imageR], queue_size=10, slop=0.5)
    ts.registerCallback(image_callback)
    rospy.spin()

def image_callback(imageL, imageR):
    br = CvBridge()
    rospy.loginfo("receiving Image")

    imageLeft = br.imgmsg_to_cv2(imageL, desired_encoding='bgr8')
    imageRight = br.imgmsg_to_cv2(imageR, desired_encoding='bgr8')

    imageLeft_new=cv.cvtColor(imageLeft, cv.COLOR_BGR2GRAY)
    imageRight_new=cv.cvtColor(imageRight, cv.COLOR_BGR2GRAY)   

    stereo = cv.StereoBM_create(numDisparities=16, blockSize=15)
    disparity = stereo.compute(imageRight_new,imageLeft_new)
    arr8 = (disparity/256).astype(np.uint8) # np.uint16 to np.uint8
    color_disparity = cv.applyColorMap(arr8, cv.COLORMAP_JET)

    rospy.loginfo("showing depth image")

    # Process images...

    pub = rospy.Publisher('/left_camera', Image, queue_size=1)

    msg = br.cv2_to_imgmsg(color_disparity, encoding='rgb8')
    pub2 = rospy.Publisher('/Right_camera', Image, queue_size=1)
    msg2 = br.cv2_to_imgmsg(color_disparity, encoding='rgb8')

    pub.publish(msg)
    pub2.publish(msg2)

if __name__ == '__main__':
    rospy.init_node('my_node')
    try:
        read_cameras()

    except rospy.ROSInterruptException:
        pass

However now i can see my /left_camera and /right_camera topics in rqt_image_view and they look like this :

this doesnt look like a disparity image to me..

[aristow1]

im closing the issue but i havent found a solution yet. if anyone has an idea let me know please