search

dusty-nv / ros_deep_learning

Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT
886 stars 258 forks source link

Video RTSP streaming #133

Open mvp-cwlin opened 8 months ago

mvp-cwlin commented 8 months ago

I am trying to use the video-output node to stream an image topic to QGroundControl using RTSP, but keep getting error messages like "streaming stopped, reason not-linked (-1)". I have also tried using rtp and save directly to file, it all worked, but I really want to use rtsp. Anything I am missing?

ros2 launch ros_deep_learning video_output.ros2.launch output:=rtsp://192.168.220.169:8554/stream output_codec:=h264 output_bitrate:=4000000 topic:=/zed/zed_node/right/image_rect_color
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-01-31-17-16-37-637263-ubuntu-7284
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [video_output-1]: process started with pid [7285]
[video_output-1] [INFO] [1706721397.845796977] [video_output]: opening video output: rtsp://192.168.220.169:8554/stream
[video_output-1] [gstreamer] initialized gstreamer, version 1.20.3.0
[video_output-1] failed to find/open file /proc/device-tree/model
[video_output-1] [gstreamer] gstEncoder -- detected board 'NVIDIA Jetson AGX Orin Developer Kit'
[video_output-1] [gstreamer] gstEncoder -- pipeline launch string:
[video_output-1] [gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! nvvidconv name=vidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc name=encoder bitrate=4000000 insert-sps-pps=1 insert-vui=1 idrinterval=30 maxperf-enable=1 ! video/x-h264 ! rtph264pay config-interval=1 name=pay0
[video_output-1] [rtsp]   waiting for RTSP server to start...
[video_output-1] [rtsp]   RTSP server started @ rtsp://ubuntu:8554
[video_output-1] [rtsp]   RTSP route added /stream @ rtsp://ubuntu:8554
[video_output-1] [video]  created gstEncoder from rtsp://192.168.220.169:8554/stream
[video_output-1] ------------------------------------------------
[video_output-1] gstEncoder video options:
[video_output-1] ------------------------------------------------
[video_output-1]   -- URI: rtsp://192.168.220.169:8554/stream
[video_output-1]      - protocol:  rtsp
[video_output-1]      - location:  192.168.220.169
[video_output-1]      - port:      8554
[video_output-1]   -- deviceType: ip
[video_output-1]   -- ioType:     output
[video_output-1]   -- codec:      H264
[video_output-1]   -- codecType:  v4l2
[video_output-1]   -- frameRate:  30
[video_output-1]   -- bitRate:    4000000
[video_output-1]   -- numBuffers: 4
[video_output-1]   -- zeroCopy:   true
[video_output-1]   -- latency     10
[video_output-1] ------------------------------------------------
[video_output-1] [OpenGL] failed to open X11 server connection.
[video_output-1] [OpenGL] failed to create X11 Window.
[video_output-1] [gstreamer] gstEncoder -- starting pipeline, transitioning to GST_STATE_PLAYING
[video_output-1] Opening in BLOCKING MODE 
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> pay0
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> encoder
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> vidconv
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> mysource
[video_output-1] [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> pay0
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> encoder
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> vidconv
[video_output-1] [gstreamer] gstreamer stream status CREATE ==> src
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> mysource
[video_output-1] [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[video_output-1] [gstreamer] gstreamer message new-clock ==> pipeline0
[video_output-1] [gstreamer] gstreamer stream status ENTER ==> src
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pay0
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> encoder
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> vidconv
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysource
[video_output-1] [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0
[video_output-1] [INFO] [1706721398.642189712] [video_output]: video_output node initialized, waiting for messages
[video_output-1] [INFO] [1706721398.731179354] [video_output]: allocated CUDA memory for 640x360 image conversion
[video_output-1] [cuda]   allocated 2 ring buffers (345600 bytes each, 691200 bytes total)
[video_output-1] [gstreamer] gstEncoder -- new caps: video/x-raw, width=640, height=360, format=(string)I420, framerate=30/1
[video_output-1] NvMMLiteOpen : Block : BlockType = 4 
[video_output-1] ===== NvVideo: NVENC =====
[video_output-1] NvMMLiteBlockCreate : Block : BlockType = 4 
[video_output-1] H264: Profile = 66, Level = 0 
[video_output-1] NVMEDIA: Need to set EMC bandwidth : 126000 
[video_output-1] NvVideo: bBlitMode is set to TRUE 
[video_output-1] [gstreamer] gstreamer message latency ==> encoder
[video_output-1] [gstreamer] gstreamer stream status CREATE ==> src
[video_output-1] [gstreamer] gstreamer stream status ENTER ==> src
[video_output-1] [gstreamer] gstreamer message qos ==> encoder
[video_output-1] [gstreamer] gstreamer mysource ERROR Internal data stream error.
[video_output-1] [gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstAppSrc:mysource:
[video_output-1] streaming stopped, reason not-linked (-1)
[video_output-1] [gstreamer] gstEncoder -- pipeline full, skipping frame 25 (640x360, 345600 bytes)
[video_output-1] [gstreamer] gstEncoder -- pipeline full, skipping frame 50 (640x360, 345600 bytes)
mvp-cwlin commented 8 months ago

while the previous ros node is running, 'video-viewer' was used on the same machine to check if it is receiving anything.

video-viewer rtsp://192.168.220.169:8554/stream my_video10.mp4
[gstreamer] initialized gstreamer, version 1.20.3.0
[gstreamer] gstDecoder -- creating decoder for 192.168.220.169
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 

(video-viewer:21894): GStreamer-CRITICAL **: 19:56:43.745: gst_debug_log_valist: assertion 'category != NULL' failed

(video-viewer:21894): GStreamer-CRITICAL **: 19:56:43.745: gst_debug_log_valist: assertion 'category != NULL' failed

(video-viewer:21894): GStreamer-CRITICAL **: 19:56:43.745: gst_debug_log_valist: assertion 'category != NULL' failed

(video-viewer:21894): GStreamer-CRITICAL **: 19:56:43.745: gst_debug_log_valist: assertion 'category != NULL' failed
[gstreamer] gstDecoder -- discovered video resolution: 640x360  (framerate 30.000000 Hz)
[gstreamer] gstDecoder -- discovered video caps:  video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3, profile=(string)constrained-baseline, width=(int)640, height=(int)360, framerate=(fraction)30/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, colorimetry=(string)2:4:0:0, parsed=(boolean)true
[gstreamer] gstDecoder -- pipeline string:
[gstreamer] rtspsrc location=rtsp://192.168.220.169:8554/stream latency=10 ! queue ! rtph264depay ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv name=vidconv ! video/x-raw ! appsink name=mysink sync=false
[video]  created gstDecoder from rtsp://192.168.220.169:8554/stream
------------------------------------------------
gstDecoder video options:
------------------------------------------------
  -- URI: rtsp://192.168.220.169:8554/stream
     - protocol:  rtsp
     - location:  192.168.220.169
     - port:      8554
  -- deviceType: ip
  -- ioType:     input
  -- codec:      H264
  -- codecType:  v4l2
  -- width:      640
  -- height:     360
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- loop:       0
  -- latency     10
------------------------------------------------
[gstreamer] gstEncoder -- codec not specified, defaulting to H.264
failed to find/open file /proc/device-tree/model
[gstreamer] gstEncoder -- detected board 'NVIDIA Jetson AGX Orin Developer Kit'
[gstreamer] gstEncoder -- pipeline launch string:
[gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! nvvidconv name=vidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc name=encoder bitrate=4000000 maxperf-enable=1 ! video/x-h264 ! h264parse ! qtmux ! filesink location=my_video10.mp4 
[video]  created gstEncoder from file:///root/ros2_ws/my_video10.mp4
------------------------------------------------
gstEncoder video options:
------------------------------------------------
  -- URI: file:///root/ros2_ws/my_video10.mp4
     - protocol:  file
     - location:  my_video10.mp4
     - extension: mp4
  -- deviceType: file
  -- ioType:     output
  -- codec:      H264
  -- codecType:  v4l2
  -- frameRate:  30
  -- bitRate:    4000000
  -- numBuffers: 4
  -- zeroCopy:   true
------------------------------------------------
[OpenGL] failed to open X11 server connection.
[OpenGL] failed to create X11 Window.
[gstreamer] opening gstDecoder for streaming, transitioning pipeline to GST_STATE_PLAYING
Opening in BLOCKING MODE 
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter2
[gstreamer] gstreamer changed state from NULL to READY ==> vidconv
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> decoder
[gstreamer] gstreamer changed state from NULL to READY ==> rtph264depay1
[gstreamer] gstreamer changed state from NULL to READY ==> queue0
[gstreamer] gstreamer changed state from NULL to READY ==> rtspsrc0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter2
[gstreamer] gstreamer changed state from READY to PAUSED ==> vidconv
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer changed state from READY to PAUSED ==> decoder
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtph264depay1
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> queue0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer changed state from READY to PAUSED ==> rtspsrc0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer message new-clock ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> vidconv
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtph264depay1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> queue0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> rtspsrc0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer rtspsrc0 ERROR Unhandled error
[gstreamer] gstreamer Debugging info: ../gst/rtsp/gstrtspsrc.c(6795): gst_rtspsrc_send (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
Service Unavailable (503)
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstreamer message progress ==> rtspsrc0
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer
[gstreamer] gstDecoder::Capture() -- a timeout occurred waiting for the next image buffer