ros-drivers / video_stream_opencv

A package to open video streams and publish them in ROS using the opencv videocapture mechanism
227 stars 159 forks source link

Docker Error Could not capture frame on Ubuntu 18.04 #90

Closed Jendker closed 3 years ago

Jendker commented 3 years ago

I have a problem with accessing the camera in docker. The video_stream_opencv is being run in docker as a root user so the access permission to the device are not an issue. Everything works fine on docker with Ubuntu 20.04 host machine but on a different computer with Ubuntu 18.04 in docker I am getting following output if I subscribe to the created camera rostopic:

started roslaunch server http://172.18.0.2:44913/

SUMMARY
========

PARAMETERS
 * /camera0/camera0_stream/buffer_queue_size: 100
 * /camera0/camera0_stream/camera_info_url: 
 * /camera0/camera0_stream/camera_name: camera0
 * /camera0/camera0_stream/flip_horizontal: False
 * /camera0/camera0_stream/flip_vertical: False
 * /camera0/camera0_stream/fps: 30.0
 * /camera0/camera0_stream/frame_id: world
 * /camera0/camera0_stream/height: 0
 * /camera0/camera0_stream/loop_videofile: False
 * /camera0/camera0_stream/set_camera_fps: 30.0
 * /camera0/camera0_stream/start_frame: 0
 * /camera0/camera0_stream/stop_frame: -1
 * /camera0/camera0_stream/video_stream_provider: 4
 * /camera0/camera0_stream/width: 0
 * /rosdistro: noetic
 * /rosversion: 1.15.9

NODES
  /camera0/
    camera0_stream (video_stream_opencv/video_stream)

auto-starting new master
process[master]: started with pid [599]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 5ab6d9ec-8b50-11eb-846a-0242ac120002
process[rosout-1]: started with pid [609]
started core service [/rosout]
process[camera0/camera0_stream-2]: started with pid [616]
[ INFO] [1616446281.084626564]: Initializing nodelet with 12 worker threads.
[ INFO] [1616446281.231082240]: Camera name: camera0
[ INFO] [1616446281.231145549]: Provided camera_info_url: ''
[ INFO] [1616446281.231181137]: Publishing with frame_id: world
[ INFO] [1616446281.231210516]: Setting camera FPS to: 30
[ INFO] [1616446281.231237541]: Throttling to fps: 30
[ INFO] [1616446281.231265247]: Setting buffer size for capturing frames to: 100
[ INFO] [1616446281.231288768]: Flip horizontal image is: false
[ INFO] [1616446281.231311911]: Flip vertical image is: false
[ INFO] [1616446281.231342150]: Video start frame is: 0
[ INFO] [1616446281.231372180]: Video stop frame is: -1
[ INFO] [1616446289.123753215]: using default calibration URL
[ INFO] [1616446289.123886318]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml
[ INFO] [1616446289.124076802]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml]
[ WARN] [1616446289.124140005]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found.
[ INFO] [1616446289.124444183]: Opening VideoCapture with provider: /dev/video4
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ INFO] [1616446290.302166801]: Video stream provider type detected: videodevice
[ INFO] [1616446290.302266712]: Camera reports FPS: 5
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1186) setProperty OpenCV | GStreamer warning: GStreamer: unhandled property
[ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Device '/dev/video4' failed during initialization
[ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (515) startPipeline OpenCV | GStreamer warning: unable to start pipeline
[ERROR] [1616446291.852088783]: Could not capture frame (frame_counter: 0)
[ WARN] [1616446291.852207511]: Waiting for device...
[ WARN] [1616446291.952562714]: Waiting for device...
[ WARN] [1616446292.052847656]: Waiting for device...
[ WARN] [1616446292.153119626]: Waiting for device...
[ WARN] [1616446292.253407089]: Waiting for device...
[ WARN] [1616446292.353719961]: Waiting for device...
[ WARN] [1616446292.454032736]: Waiting for device...
[ WARN] [1616446292.554347708]: Waiting for device...
[ WARN] [1616446292.654681905]: Waiting for device...
[ WARN] [1616446292.754938392]: Waiting for device...
[ WARN] [1616446292.855224316]: Waiting for device...
[ WARN] [1616446292.955508793]: Waiting for device...
[ WARN] [1616446293.055815889]: Waiting for device...
[ WARN] [1616446293.156144592]: Waiting for device...
[ WARN] [1616446293.256436284]: Waiting for device...
[ WARN] [1616446293.356739560]: Waiting for device...
[ WARN] [1616446293.457051838]: Waiting for device...
[ WARN] [1616446293.557366183]: Waiting for device...
[ WARN] [1616446293.657679612]: Waiting for device...
[ WARN] [1616446293.757997069]: Waiting for device...
[ WARN] [1616446293.858312195]: Waiting for device...
[ WARN] [1616446293.958634418]: Waiting for device...
...

and sometimes after subscribing to the topic I get the following output because the device /dev/videoX disappears after I try subscribing until I reconnect the camera:

SUMMARY
========

PARAMETERS
 * /camera0/camera0_stream/buffer_queue_size: 100
 * /camera0/camera0_stream/camera_info_url: 
 * /camera0/camera0_stream/camera_name: camera0
 * /camera0/camera0_stream/flip_horizontal: False
 * /camera0/camera0_stream/flip_vertical: False
 * /camera0/camera0_stream/fps: 30.0
 * /camera0/camera0_stream/frame_id: world
 * /camera0/camera0_stream/height: 0
 * /camera0/camera0_stream/loop_videofile: False
 * /camera0/camera0_stream/set_camera_fps: 30.0
 * /camera0/camera0_stream/start_frame: 0
 * /camera0/camera0_stream/stop_frame: -1
 * /camera0/camera0_stream/video_stream_provider: 0
 * /camera0/camera0_stream/width: 0
 * /rosdistro: noetic
 * /rosversion: 1.15.9

NODES
  /camera0/
    camera0_stream (video_stream_opencv/video_stream)

auto-starting new master
process[master]: started with pid [307]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to ad187b6c-8b5c-11eb-b2bc-0242ac120002
process[rosout-1]: started with pid [317]
started core service [/rosout]
process[camera0/camera0_stream-2]: started with pid [324]
[ INFO] [1616451573.239341205]: Initializing nodelet with 12 worker threads.
[ INFO] [1616451573.380970996]: Camera name: camera0
[ INFO] [1616451573.381068189]: Provided camera_info_url: ''
[ INFO] [1616451573.381116147]: Publishing with frame_id: world
[ INFO] [1616451573.381156614]: Setting camera FPS to: 30
[ INFO] [1616451573.381193775]: Throttling to fps: 30
[ INFO] [1616451573.381251976]: Setting buffer size for capturing frames to: 100
[ INFO] [1616451573.381303064]: Flip horizontal image is: false
[ INFO] [1616451573.381349959]: Flip vertical image is: false
[ INFO] [1616451573.381394840]: Video start frame is: 0
[ INFO] [1616451573.381434548]: Video stop frame is: -1
[ INFO] [1616451577.881623925]: using default calibration URL
[ INFO] [1616451577.881734196]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml
[ INFO] [1616451577.881938309]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml]
[ WARN] [1616451577.882080689]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found.
[ INFO] [1616451577.882399145]: Opening VideoCapture with provider: /dev/video0
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Could not open device '/dev/video0' for reading and writing.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0] global ../modules/videoio/src/cap_v4l.cpp (887) open VIDEOIO(V4L2:/dev/video0): can't open camera by index
[ INFO] [1616451748.986824755]: Video stream provider type detected: videodevice
[ INFO] [1616451748.986912183]: Backend can't provide camera FPS information
[ERROR] [1616451748.986960284]: Could not open the stream

I am using the following options for running docker container:

version: "2.4"
services:
  robonet:
    build:
      context: ..
      dockerfile: ./docker/Dockerfile
    image: robonet
    container_name: robonet
    working_dir: /root
    environment:
      - DISPLAY=:0
    runtime: nvidia
    volumes:
       - /dev:/dev
    command: bash
    stdin_open: true
    tty: true
    device_cgroup_rules:
      - 'c *:* rmw'

I am able to access the camera on the machine without any problems though when I tried the x11docker, so I assume it is video_stream_opencv or (maybe?) GStreamer specific.

Please let me know if any additional information would be helpful.

awesomebytes commented 3 years ago

My fist try would be execute docker with --privileged and remove the volume mounting rule to really discard it's a docker thing. Note that you are exposing /dev as a volume and it's actually a device (so it needs to be mounted differently AFAIK).

If it doesn't work, I'd try to execute the script on the script folders that tests just the opencv subscriber (no ROS or anything fancy in it, if you don't have X enabled for that container, you'll need to comment out the showing the image from the script but you could just print that you got a frame). If that doesn't work... Well I guess it doesn't work for you setup? That said, I used the node in docker on a webcam and it worked a while ago.

You can also try to install some other tool that accesses the webcam to see if they can access it. cheese comes to mind (but again, you'll need X/graphical system enabled to be able to open the window).

Sorry for the brevity, writing from my phone.

On Tue, Mar 23, 2021, 08:06 Jędrzej Beniamin Orbik @.***> wrote:

I have a problem with accessing the camera in docker. The video_stream_opencv is being run in docker as a root user so the access permission to the device are not an issue. Everything works fine on docker with Ubuntu 20.04 host machine but on a different computer with Ubuntu 18.04 in docker I am getting following output:

started roslaunch server http://172.18.0.2:44913/

SUMMARY

PARAMETERS

  • /camera0/camera0_stream/buffer_queue_size: 100
  • /camera0/camera0_stream/camera_info_url:
  • /camera0/camera0_stream/camera_name: camera0
  • /camera0/camera0_stream/flip_horizontal: False
  • /camera0/camera0_stream/flip_vertical: False
  • /camera0/camera0_stream/fps: 30.0
  • /camera0/camera0_stream/frame_id: world
  • /camera0/camera0_stream/height: 0
  • /camera0/camera0_stream/loop_videofile: False
  • /camera0/camera0_stream/set_camera_fps: 30.0
  • /camera0/camera0_stream/start_frame: 0
  • /camera0/camera0_stream/stop_frame: -1
  • /camera0/camera0_stream/video_stream_provider: 4
  • /camera0/camera0_stream/width: 0
  • /rosdistro: noetic
  • /rosversion: 1.15.9

NODES /camera0/ camera0_stream (video_stream_opencv/video_stream)

auto-starting new master process[master]: started with pid [599] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 5ab6d9ec-8b50-11eb-846a-0242ac120002 process[rosout-1]: started with pid [609] started core service [/rosout] process[camera0/camera0_stream-2]: started with pid [616] [ INFO] [1616446281.084626564]: Initializing nodelet with 12 worker threads. [ INFO] [1616446281.231082240]: Camera name: camera0 [ INFO] [1616446281.231145549]: Provided camera_info_url: '' [ INFO] [1616446281.231181137]: Publishing with frame_id: world [ INFO] [1616446281.231210516]: Setting camera FPS to: 30 [ INFO] [1616446281.231237541]: Throttling to fps: 30 [ INFO] [1616446281.231265247]: Setting buffer size for capturing frames to: 100 [ INFO] [1616446281.231288768]: Flip horizontal image is: false [ INFO] [1616446281.231311911]: Flip vertical image is: false [ INFO] [1616446281.231342150]: Video start frame is: 0 [ INFO] [1616446281.231372180]: Video stop frame is: -1 [ INFO] [1616446289.123753215]: using default calibration URL [ INFO] [1616446289.123886318]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml [ INFO] [1616446289.124076802]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml] [ WARN] [1616446289.124140005]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found. [ INFO] [1616446289.124444183]: Opening VideoCapture with provider: /dev/video4 [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1 [ INFO] [1616446290.302166801]: Video stream provider type detected: videodevice [ INFO] [1616446290.302266712]: Camera reports FPS: 5 [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1186) setProperty OpenCV | GStreamer warning: GStreamer: unhandled property [ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Device '/dev/video4' failed during initialization [ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (515) startPipeline OpenCV | GStreamer warning: unable to start pipeline [ERROR] [1616446291.852088783]: Could not capture frame (frame_counter: 0) [ WARN] [1616446291.852207511]: Waiting for device... [ WARN] [1616446291.952562714]: Waiting for device... [ WARN] [1616446292.052847656]: Waiting for device... [ WARN] [1616446292.153119626]: Waiting for device... [ WARN] [1616446292.253407089]: Waiting for device... [ WARN] [1616446292.353719961]: Waiting for device... [ WARN] [1616446292.454032736]: Waiting for device... [ WARN] [1616446292.554347708]: Waiting for device... [ WARN] [1616446292.654681905]: Waiting for device... [ WARN] [1616446292.754938392]: Waiting for device... [ WARN] [1616446292.855224316]: Waiting for device... [ WARN] [1616446292.955508793]: Waiting for device... [ WARN] [1616446293.055815889]: Waiting for device... [ WARN] [1616446293.156144592]: Waiting for device... [ WARN] [1616446293.256436284]: Waiting for device... [ WARN] [1616446293.356739560]: Waiting for device... [ WARN] [1616446293.457051838]: Waiting for device... [ WARN] [1616446293.557366183]: Waiting for device... [ WARN] [1616446293.657679612]: Waiting for device... [ WARN] [1616446293.757997069]: Waiting for device... [ WARN] [1616446293.858312195]: Waiting for device... [ WARN] [1616446293.958634418]: Waiting for device... ...

I am using the following options for running docker container:

version: "2.4" services: robonet: build: context: .. dockerfile: ./docker/Dockerfile image: robonet container_name: robonet working_dir: /root environment:

  • DISPLAY=:0 runtime: nvidia volumes:
    • /dev:/dev command: bash stdin_open: true tty: true device_cgroup_rules:
  • 'c : rmw'

I am able to access the camera on the machine without any problems though when I tried the (https://github.com/mviereck/x11docker)[x11docker], so I assume it is video_stream_opencv or (maybe?) GStreamer specific.

Please let me know if any additional information would be helpful.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ros-drivers/video_stream_opencv/issues/90, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEK5DTDR4V3BA63AOJOI3TE6WOTANCNFSM4ZT4LGFQ .

Jendker commented 3 years ago

Thank you for your answer. It still fails with the script provided. I am getting:

Trying to open resource:  /dev/video

(python3:2454): GStreamer-CRITICAL **: 21:01:29.729: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: no source element for URI "/dev/video"
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ERROR:0] global ../modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.2.0) ../modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /dev/video in function 'icvExtractPattern'

Error opening resource: /dev/video
Maybe opencv VideoCapture can't open it

And yes, it works just fine with cheese and X11 which I have tried out using the x11docker, the same docker image on the same machine, so it does not seem to be docker specific issue. Do you think there are any additional debugging steps I could try out?

awesomebytes commented 3 years ago

/dev/video looks weird, shouldn't it be /dev/video0 ?

On Tue, Mar 23, 2021, 15:06 Jędrzej Beniamin Orbik @.***> wrote:

Thank you for your answer. It still fails with the script provided. I am getting:

Trying to open resource: /dev/video

(python3:2454): GStreamer-CRITICAL **: 21:01:29.729: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: no source element for URI "/dev/video" [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created [ERROR:0] global ../modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.2.0) ../modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /dev/video in function 'icvExtractPattern'

Error opening resource: /dev/video Maybe opencv VideoCapture can't open it

And yes, it works just fine with cheese and X11 which I have tried out using the x11docker https://github.com/mviereck/x11docker, the same docker image on the same machine. Do you think there are any additional debugging steps I could try out?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ros-drivers/video_stream_opencv/issues/90#issuecomment-804600772, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEK5FHVE5ZBC7AUFJJR2DTFAHVJANCNFSM4ZT4LGFQ .

Jendker commented 3 years ago

Well, that's when you know you should get some rest... Thank you for your thoroughness when looking into the details. Yes, now the script works just fine. I am getting the arrays with the data.

Trying to open resource:  /dev/video0
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Correctly opened resource, starting to show feed.
< here I can see the data I am printing instead of showing with cv2.imshow>

I have double checked the output of the roslaunch and the script is using the correct device (/dev/video0) as expected:

started roslaunch server http://172.18.0.2:43763/

SUMMARY
========

PARAMETERS
 * /camera0/camera0_stream/buffer_queue_size: 100
 * /camera0/camera0_stream/camera_info_url: 
 * /camera0/camera0_stream/camera_name: camera0
 * /camera0/camera0_stream/flip_horizontal: False
 * /camera0/camera0_stream/flip_vertical: False
 * /camera0/camera0_stream/fps: 30.0
 * /camera0/camera0_stream/frame_id: world
 * /camera0/camera0_stream/height: 0
 * /camera0/camera0_stream/loop_videofile: False
 * /camera0/camera0_stream/set_camera_fps: 30.0
 * /camera0/camera0_stream/start_frame: 0
 * /camera0/camera0_stream/stop_frame: -1
 * /camera0/camera0_stream/video_stream_provider: 0
 * /camera0/camera0_stream/width: 0
 * /rosdistro: noetic
 * /rosversion: 1.15.9

NODES
  /camera0/
    camera0_stream (video_stream_opencv/video_stream)

auto-starting new master
process[master]: started with pid [2490]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to a4417122-8b90-11eb-ab12-0242ac120002
process[rosout-1]: started with pid [2500]
started core service [/rosout]
process[camera0/camera0_stream-2]: started with pid [2507]
[ INFO] [1616473892.122017524]: Initializing nodelet with 12 worker threads.
[ INFO] [1616473892.269017837]: Camera name: camera0
[ INFO] [1616473892.269092387]: Provided camera_info_url: ''
[ INFO] [1616473892.269122316]: Publishing with frame_id: world
[ INFO] [1616473892.269161280]: Setting camera FPS to: 30
[ INFO] [1616473892.269195510]: Throttling to fps: 30
[ INFO] [1616473892.269228569]: Setting buffer size for capturing frames to: 100
[ INFO] [1616473892.269261826]: Flip horizontal image is: false
[ INFO] [1616473892.269297414]: Flip vertical image is: false
[ INFO] [1616473892.269353214]: Video start frame is: 0
[ INFO] [1616473892.269399203]: Video stop frame is: -1
[ INFO] [1616473904.271719238]: using default calibration URL
[ INFO] [1616473904.271822131]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml
[ INFO] [1616473904.271976293]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml]
[ WARN] [1616473904.272118591]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found.
[ INFO] [1616473904.272412296]: Opening VideoCapture with provider: /dev/video0
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ INFO] [1616473905.347854572]: Video stream provider type detected: videodevice
[ INFO] [1616473905.347941888]: Camera reports FPS: 5
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1186) setProperty OpenCV | GStreamer warning: GStreamer: unhandled property
[ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Device '/dev/video0' failed during initialization
[ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (515) startPipeline OpenCV | GStreamer warning: unable to start pipeline
[ERROR] [1616473906.894099679]: Could not capture frame (frame_counter: 0)
[ WARN] [1616473906.894272735]: Waiting for device...
[ WARN] [1616473906.994632578]: Waiting for device...
[ WARN] [1616473907.094821212]: Waiting for device...
[ WARN] [1616473907.195127410]: Waiting for device...

Does it mean that it is ROS specific or do we have more moving parts which I should debug?

awesomebytes commented 3 years ago

Seeing this error:

Embedded video playback halted; module v4l2src0 reported: Device '/dev/video0' failed during initialization

I see that the backend opencv is using is v4l2 via gstreamer.

(Really this ROS package is just a wrapper of opencv, that wraps whatever available backend if available in the system).

I'd try to debug v4l2 or gstreamer. You could use gscam to publish your camera too.

On Tue, Mar 23, 2021, 15:36 Jędrzej Beniamin Orbik @.***> wrote:

Well, that's when you know you should get some rest... Thank you for your thoroughness when looking into the details. Yes, now the script works just fine. I am getting the arrays with the data. I have double checked the output of the roslaunch and the script is using the correct device (/dev/video0) as expected:

started roslaunch server http://172.18.0.2:43763/

SUMMARY

PARAMETERS

  • /camera0/camera0_stream/buffer_queue_size: 100
  • /camera0/camera0_stream/camera_info_url:
  • /camera0/camera0_stream/camera_name: camera0
  • /camera0/camera0_stream/flip_horizontal: False
  • /camera0/camera0_stream/flip_vertical: False
  • /camera0/camera0_stream/fps: 30.0
  • /camera0/camera0_stream/frame_id: world
  • /camera0/camera0_stream/height: 0
  • /camera0/camera0_stream/loop_videofile: False
  • /camera0/camera0_stream/set_camera_fps: 30.0
  • /camera0/camera0_stream/start_frame: 0
  • /camera0/camera0_stream/stop_frame: -1
  • /camera0/camera0_stream/video_stream_provider: 0
  • /camera0/camera0_stream/width: 0
  • /rosdistro: noetic
  • /rosversion: 1.15.9

NODES /camera0/ camera0_stream (video_stream_opencv/video_stream)

auto-starting new master process[master]: started with pid [2490] ROS_MASTER_URI=http://localhost:11311

setting /run_id to a4417122-8b90-11eb-ab12-0242ac120002 process[rosout-1]: started with pid [2500] started core service [/rosout] process[camera0/camera0_stream-2]: started with pid [2507] [ INFO] [1616473892.122017524]: Initializing nodelet with 12 worker threads. [ INFO] [1616473892.269017837]: Camera name: camera0 [ INFO] [1616473892.269092387]: Provided camera_info_url: '' [ INFO] [1616473892.269122316]: Publishing with frame_id: world [ INFO] [1616473892.269161280]: Setting camera FPS to: 30 [ INFO] [1616473892.269195510]: Throttling to fps: 30 [ INFO] [1616473892.269228569]: Setting buffer size for capturing frames to: 100 [ INFO] [1616473892.269261826]: Flip horizontal image is: false [ INFO] [1616473892.269297414]: Flip vertical image is: false [ INFO] [1616473892.269353214]: Video start frame is: 0 [ INFO] [1616473892.269399203]: Video stop frame is: -1 [ INFO] [1616473904.271719238]: using default calibration URL [ INFO] [1616473904.271822131]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml [ INFO] [1616473904.271976293]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml] [ WARN] [1616473904.272118591]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found. [ INFO] [1616473904.272412296]: Opening VideoCapture with provider: /dev/video0 [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1 [ INFO] [1616473905.347854572]: Video stream provider type detected: videodevice [ INFO] [1616473905.347941888]: Camera reports FPS: 5 [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1186) setProperty OpenCV | GStreamer warning: GStreamer: unhandled property [ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (1758) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Device '/dev/video0' failed during initialization [ WARN:1] global ../modules/videoio/src/cap_gstreamer.cpp (515) startPipeline OpenCV | GStreamer warning: unable to start pipeline [ERROR] [1616473906.894099679]: Could not capture frame (frame_counter: 0) [ WARN] [1616473906.894272735]: Waiting for device... [ WARN] [1616473906.994632578]: Waiting for device... [ WARN] [1616473907.094821212]: Waiting for device... [ WARN] [1616473907.195127410]: Waiting for device...

Does it mean that it is ROS specific or do we have more moving parts which I should debug?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ros-drivers/video_stream_opencv/issues/90#issuecomment-804610344, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEK5BZY6YRQ3KNIIQPD5TTFALDTANCNFSM4ZT4LGFQ .

awesomebytes commented 3 years ago

@Jendker if you are able to share your docker image, I could give it a go. I just tried video_stream_opencv on my laptop in a docker container that had some ROS installed (with --privileged --net host as the only docker flags, the second one just to use my host rqt_image_view to visualize my webcam), and it worked.

Again, it may have to do with the backend OpenCV is using in your case, maybe check what's available on your image: https://docs.opencv.org/3.2.0/d0/da7/videoio_overview.html

You could call the OpenCV getBuildInformation method to get a list of what was compiled for that OpenCV.

Jendker commented 3 years ago

Thank you! The docker image I am using is jendker/robonetv2 (https://hub.docker.com/repository/docker/jendker/robonetv2) and I am running video_stream_opencv with

roslaunch video_stream_opencv my.launch

This works fine on a Ubuntu 20.04 host, but failed on 2 different 18.04 host machines (all run withing docker).

awesomebytes commented 3 years ago

I've tried that image and it worked for me without a problem on my webcam. My host system is an ubuntu 18.04, but that should not matter (as that's all the point of using docker).

What may matter is the kernel version of the host, mine is: Linux xps 5.4.0-65-generic #73~18.04.1-Ubuntu SMP Tue Jan 19 09:02:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux (from uname -a).

I think it has to do with your hardware, unfortunately.

Also my output from launching my.launch (note I launched the docker image with: docker run --privileged --net host -it jendker/robonetv2:latest bash):

root@xps:~/interbotix_ws/src/video_stream_opencv/launch# roslaunch my.launch 
... logging to /root/.ros/log/df6792c0-8bdd-11eb-b9e9-001500e8fc83/roslaunch-xps-254.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://192.168.20.29:41571/

SUMMARY
========

PARAMETERS
 * /camera0/camera0_stream/buffer_queue_size: 100
 * /camera0/camera0_stream/camera_info_url: 
 * /camera0/camera0_stream/camera_name: camera0
 * /camera0/camera0_stream/flip_horizontal: False
 * /camera0/camera0_stream/flip_vertical: False
 * /camera0/camera0_stream/fps: 30.0
 * /camera0/camera0_stream/frame_id: world
 * /camera0/camera0_stream/height: 0
 * /camera0/camera0_stream/loop_videofile: False
 * /camera0/camera0_stream/set_camera_fps: 30.0
 * /camera0/camera0_stream/start_frame: 0
 * /camera0/camera0_stream/stop_frame: -1
 * /camera0/camera0_stream/video_stream_provider: 0
 * /camera0/camera0_stream/width: 0
 * /rosdistro: noetic
 * /rosversion: 1.15.9

NODES
  /camera0/
    camera0_stream (video_stream_opencv/video_stream)

auto-starting new master
process[master]: started with pid [262]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to df6792c0-8bdd-11eb-b9e9-001500e8fc83
process[rosout-1]: started with pid [272]
started core service [/rosout]
process[camera0/camera0_stream-2]: started with pid [279]
[ INFO] [1616507062.794436289]: Initializing nodelet with 4 worker threads.
[ INFO] [1616507062.965203058]: Camera name: camera0
[ INFO] [1616507062.965254005]: Provided camera_info_url: ''
[ INFO] [1616507062.965291566]: Publishing with frame_id: world
[ INFO] [1616507062.965327522]: Setting camera FPS to: 30
[ INFO] [1616507062.965399367]: Throttling to fps: 30
[ INFO] [1616507062.965423638]: Setting buffer size for capturing frames to: 100
[ INFO] [1616507062.965446390]: Flip horizontal image is: false
[ INFO] [1616507062.965468278]: Flip vertical image is: false
[ INFO] [1616507062.965497222]: Video start frame is: 0
[ INFO] [1616507062.965518451]: Video stop frame is: -1
[ INFO] [1616507078.403954685]: using default calibration URL
[ INFO] [1616507078.404169491]: camera calibration URL: file:///root/.ros/camera_info/camera0.yaml
[ INFO] [1616507078.404540858]: Unable to open camera calibration file [/root/.ros/camera_info/camera0.yaml]
[ WARN] [1616507078.404694087]: Camera calibration file /root/.ros/camera_info/camera0.yaml not found.
[ INFO] [1616507078.405166851]: Opening VideoCapture with provider: /dev/video0
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ INFO] [1616507078.745822939]: Video stream provider type detected: videodevice
[ INFO] [1616507078.745941280]: Camera reports FPS: 30
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1186) setProperty OpenCV | GStreamer warning: GStreamer: unhandled property
[ WARN] [1616507079.184967322]: No calibration file given, publishing a reasonable default camera info.
[ INFO] [1616507079.185268093]: The image width is: 640
[ INFO] [1616507079.185403558]: The image height is: 480

Also I see you are using noetic with OpenCV 4.2, I'm happy to see that video_stream_opencv still works there untouched :)

Good luck!

Jendker commented 3 years ago

Thank you for your input! I’ll update the issue as soon as I get more details about my case.

Jendker commented 3 years ago

Just last question: would you be able to say shortly what does the script do different than the nodelet launched with roslaunch file so that I can pin down what my issue may be?

awesomebytes commented 3 years ago

The script uses the python opencv which may be (most probably is) different from your system/ROS opencv.

That's all i can think of that's different. Otherwise, they call the same API to grab an image.

On Wed, Mar 24, 2021, 01:00 Jędrzej Beniamin Orbik @.***> wrote:

Just last question: would you be able to say shortly what does the script do different than the nodelet launched with roslaunch file so that I can pin down what my issue may be?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ros-drivers/video_stream_opencv/issues/90#issuecomment-804926101, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANEK5BMPGFOPZUER5EJ6M3TFCNGPANCNFSM4ZT4LGFQ .

Jendker commented 3 years ago

Thank you for your help @awesomebytes ! I was able to get around the issue by using the python calls for now, even though it is rather minimal version comparing to what the C++ offers. I have included the changes in my fork: https://github.com/Jendker/video_stream_opencv Let me know if opening a PR with these changes would be somehow beneficial for this repo as well.