How to perform simple camera operations

[agutierrez0]

Hello,

I am working on a project an intern last worked on and want to familiarize myself with the camera on this stretch-re1 device. I have tried creating a simple python script using openCV however I keep getting errors saying that I cannot access the camera via index. I have searched through the forums and found this link: https://forum.hello-robot.com/t/recording-video-from-the-camera/170 but the script the poster provided has not worked for me.

I simply want to use openCV (or any alternative) to capture an image, utilize that image, and then operate the robot.

Any help would be greatly appreciated.

[hello-binit]

Hi @agutierrez0, happy to help. Would you paste here the python script you've created and the errors you're receiving?

Additionally, please note that this repository is for our ROS bindings. In the future, a issue regarding working with Stretch from Python directly would be better suited for our forum.

[agutierrez0]

Hi @hello-binit , apologies for the late response.

I was using the following code in efforts to get a feed from the camera:

import pyrealsense2 as rs
import numpy as np
import cv2

# Configure depth and color streams
pipeline = rs.pipeline()
config = rs.config()

# Get device product line for setting a supporting resolution
pipeline_wrapper = rs.pipeline_wrapper(pipeline)
pipeline_profile = config.resolve(pipeline_wrapper)
device = pipeline_profile.get_device()
device_product_line = str(device.get_info(rs.camera_info.product_line))

# Setting up config for D435i
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)

pipeline.start(config)

try:
    while True:
        frames = pipeline.wait_for_frames()
        depth_frame = frames.get_depth_frame()
        color_frame = frames.get_color_frame()
        if not depth_frame or not color_frame:
            continue

        # Convert images to numpy arrays
        depth_image = np.asanyarray(depth_frame.get_data())
        color_image = np.asanyarray(color_frame.get_data())

        cv2.imshow('color_image', color_image)
        cv2.waitKey(1)
finally:
    pipeline.stop()

When running it, I wasn't getting a window showing the feed. I decided to unplug and replug the two USBs that were connected and gave it one last restart. When trying it again after rebooting, it worked! I noticed that the reason it wasn't working before was because I had tried using the ROS CLI before running the python script. If i do any roslaunch commands, I no longer get a feed from the video.

Happy to say that the issue is resolved on my end. Thanks for the follow up.

[hello-binit]

Glad to hear it! Also FYI, we've seen the same issue where the pyrealsense2 interface to the camera doesn't work after using the ROS camera drivers. Fortunately, the issue seems to be fixed with the latest versions of Realsense software. If fixing the issue becomes a priority, upgrading your robot's software should solve the problem.