Closed hunterwn closed 4 weeks ago
When I try to create an rtsp stream using the nano_llm docker image I get a GStreamer internal data stream error. WebRTC works fine with the same video device.
This is the command I'm running:
jetson-containers run $(autotag nano_llm) \ python3 -m nano_llm.agents.video_stream \ --video-input /dev/video0 \ --video-output rtsp://@:1234/output
Here's the full output of the command:
Namespace(packages=['nano_llm'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False) -- L4T_VERSION=36.3.0 JETPACK_VERSION=6.0 CUDA_VERSION=12.2 -- Finding compatible container image for ['nano_llm'] nano_llm:24.5.1-r36.3.0 + docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /mnt/nvme/projects/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb --device /dev/video0 --device /dev/video1 --device /dev/video2 --device /dev/video3 --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-3 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-6 --device /dev/i2c-7 --device /dev/i2c-8 --device /dev/i2c-9 nano_llm:24.5.1-r36.3.0 python3 -m nano_llm.agents.video_stream --video-input /dev/video0 --video-output rtsp://@:1234/output /usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead. warnings.warn( (gst-plugin-scanner:58): GLib-GObject-WARNING **: 18:54:02.837: cannot register existing type 'GstRtpSrc' (gst-plugin-scanner:58): GLib-GObject-CRITICAL **: 18:54:02.837: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-plugin-scanner:58): GLib-CRITICAL **: 18:54:02.837: g_once_init_leave: assertion 'result != 0' failed (gst-plugin-scanner:58): GStreamer-CRITICAL **: 18:54:02.837: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed (gst-plugin-scanner:58): GLib-GObject-WARNING **: 18:54:02.837: cannot register existing type 'GstRtpSink' (gst-plugin-scanner:58): GLib-GObject-CRITICAL **: 18:54:02.837: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed (gst-plugin-scanner:58): GLib-CRITICAL **: 18:54:02.837: g_once_init_leave: assertion 'result != 0' failed (gst-plugin-scanner:58): GStreamer-CRITICAL **: 18:54:02.837: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed (gst-plugin-scanner:58): GStreamer-WARNING **: 18:54:03.297: adding type GstEvent multiple times sh: 1: lsmod: not found sh: 1: modprobe: not found [gstreamer] initialized gstreamer, version 1.20.3.0 [gstreamer] gstCamera -- attempting to create device v4l2:///dev/video0 [gstreamer] gstCamera -- found v4l2 device: Logitech BRIO [gstreamer] v4l2-proplist, device.path=(string)/dev/video2, udev-probed=(boolean)false, device.api=(string)v4l2, v4l2.device.driver=(string)uvcvideo, v4l2.device.card=(string)"Logitech\ BRIO", v4l2.device.bus_info=(string)usb-3610000.usb-3.2, v4l2.device.version=(uint)331656, v4l2.device.capabilities=(uint)2225078273, v4l2.device.device_caps=(uint)69206017; [gstreamer] gstCamera -- found v4l2 device: Logitech BRIO [gstreamer] v4l2-proplist, device.path=(string)/dev/video0, udev-probed=(boolean)false, device.api=(string)v4l2, v4l2.device.driver=(string)uvcvideo, v4l2.device.card=(string)"Logitech\ BRIO", v4l2.device.bus_info=(string)usb-3610000.usb-3.2, v4l2.device.version=(uint)331656, v4l2.device.capabilities=(uint)2225078273, v4l2.device.device_caps=(uint)69206017; [gstreamer] gstCamera -- found 43 caps for v4l2 device /dev/video0 [gstreamer] [0] video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [1] video/x-raw, format=(string)YUY2, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [2] video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [3] video/x-raw, format=(string)YUY2, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [4] video/x-raw, format=(string)YUY2, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [5] video/x-raw, format=(string)YUY2, width=(int)800, height=(int)600, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [6] video/x-raw, format=(string)YUY2, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [7] video/x-raw, format=(string)YUY2, width=(int)800, height=(int)448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [8] video/x-raw, format=(string)YUY2, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [9] video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [10] video/x-raw, format=(string)YUY2, width=(int)440, height=(int)440, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; [gstreamer] [11] video/x-raw, format=(string)YUY2, width=(int)480, height=(int)270, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [12] video/x-raw, format=(string)YUY2, width=(int)340, height=(int)340, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1; [gstreamer] [13] video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [14] video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [15] video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [16] video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [17] video/x-raw, format=(string)YUY2, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [18] video/x-raw, format=(string)YUY2, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [19] video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [20] video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [21] video/x-raw, format=(string)NV12, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [22] video/x-raw, format=(string)NV12, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [23] image/jpeg, width=(int)4096, height=(int)2160, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [24] image/jpeg, width=(int)3840, height=(int)2160, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [25] image/jpeg, width=(int)2560, height=(int)1440, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [26] image/jpeg, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 60/1, 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [27] image/jpeg, width=(int)1600, height=(int)896, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [28] image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 90/1, 60/1, 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [29] image/jpeg, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [30] image/jpeg, width=(int)960, height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [31] image/jpeg, width=(int)800, height=(int)600, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [32] image/jpeg, width=(int)848, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [33] image/jpeg, width=(int)800, height=(int)448, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [34] image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 120/1, 90/1, 60/1, 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [35] image/jpeg, width=(int)640, height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [36] image/jpeg, width=(int)480, height=(int)270, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [37] image/jpeg, width=(int)424, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [38] image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [39] image/jpeg, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [40] image/jpeg, width=(int)320, height=(int)180, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [41] image/jpeg, width=(int)176, height=(int)144, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] [42] image/jpeg, width=(int)160, height=(int)120, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 24/1, 20/1, 15/1, 10/1, 15/2, 5/1 }; [gstreamer] gstCamera -- selected device profile: codec=raw format=yuyv width=1280 height=720 framerate=30 [gstreamer] gstCamera pipeline string: [gstreamer] v4l2src device=/dev/video0 do-timestamp=true ! video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=30/1 ! appsink name=mysink sync=false [gstreamer] gstCamera successfully created device v4l2:///dev/video0 [video] created gstCamera from v4l2:///dev/video0 ------------------------------------------------ gstCamera video options: ------------------------------------------------ -- URI: v4l2:///dev/video0 - protocol: v4l2 - location: /dev/video0 -- deviceType: v4l2 -- ioType: input -- codec: raw -- codecType: cpu -- width: 1280 -- height: 720 -- frameRate: 30 -- numBuffers: 4 -- zeroCopy: true -- flipMethod: none -- sslCert /etc/ssl/private/localhost.cert.pem -- sslKey /etc/ssl/private/localhost.key.pem ------------------------------------------------ [gstreamer] gstEncoder -- codec not specified, defaulting to H.264 failed to find/open file /proc/device-tree/model [gstreamer] gstEncoder -- detected board 'NVIDIA Jetson AGX Orin Developer Kit' [gstreamer] gstEncoder -- pipeline launch string: [gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! nvvidconv name=vidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc name=encoder bitrate=4000000 insert-sps-pps=1 insert-vui=1 idrinterval=30 maxperf-enable=1 ! video/x-h264 ! rtph264pay config-interval=1 name=pay0 sh: 1: lsmod: not found sh: 1: modprobe: not found [rtsp] waiting for RTSP server to start... [rtsp] RTSP server started @ rtsp://ubuntu:1234 [rtsp] RTSP route added /output @ rtsp://ubuntu:1234 [video] created gstEncoder from rtsp://@:1234/output ------------------------------------------------ gstEncoder video options: ------------------------------------------------ -- URI: rtsp://@:1234/output - protocol: rtsp - location: 0.0.0.0 - port: 1234 -- deviceType: ip -- ioType: output -- codec: H264 -- codecType: v4l2 -- frameRate: 30 -- bitRate: 4000000 -- numBuffers: 4 -- zeroCopy: true -- latency 10 -- sslCert /etc/ssl/private/localhost.cert.pem -- sslKey /etc/ssl/private/localhost.key.pem ------------------------------------------------ [gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING [gstreamer] gstreamer changed state from NULL to READY ==> mysink [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0 [gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0 [gstreamer] gstreamer changed state from NULL to READY ==> pipeline0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0 [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0 [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0 [gstreamer] gstreamer message new-clock ==> pipeline0 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0 [gstreamer] gstreamer message stream-start ==> pipeline0 18:54:04 | SUCCESS | VideoStream - system ready [gstreamer] gstCamera -- onPreroll [gstreamer] gstBufferManager recieve caps: video/x-raw, format=(string)YUY2, width=(int)1280, height=(int)720, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)2:4:5:1 [gstreamer] gstBufferManager -- recieved first frame, codec=raw format=yuyv width=1280 height=720 size=1843200 [cuda] allocated 4 ring buffers (1843200 bytes each, 7372800 bytes total) [cuda] allocated 4 ring buffers (8 bytes each, 32 bytes total) [gstreamer] gstreamer changed state from READY to PAUSED ==> mysink [gstreamer] gstreamer message async-done ==> pipeline0 [gstreamer] gstreamer message latency ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0 [gstreamer] gstreamer message qos ==> v4l2src0 [cuda] allocated 4 ring buffers (2764800 bytes each, 11059200 bytes total) [cuda] allocated 2 ring buffers (1382400 bytes each, 2764800 bytes total) [gstreamer] gstEncoder -- starting pipeline, transitioning to GST_STATE_PLAYING Opening in BLOCKING MODE [gstreamer] gstreamer changed state from NULL to READY ==> pay0 [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter2 [gstreamer] gstreamer changed state from NULL to READY ==> encoder [gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1 [gstreamer] gstreamer changed state from NULL to READY ==> vidconv [gstreamer] gstreamer changed state from NULL to READY ==> mysource [gstreamer] gstreamer changed state from NULL to READY ==> pipeline1 [gstreamer] gstreamer changed state from READY to PAUSED ==> pay0 [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter2 [gstreamer] gstreamer changed state from READY to PAUSED ==> encoder [gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1 [gstreamer] gstreamer changed state from READY to PAUSED ==> vidconv [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer changed state from READY to PAUSED ==> mysource [gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline1 [gstreamer] gstreamer message new-clock ==> pipeline1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pay0 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter2 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> encoder [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1 [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> vidconv [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysource [gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline1 [gstreamer] gstreamer stream status ENTER ==> src [gstreamer] gstreamer message qos ==> v4l2src0 [gstreamer] gstEncoder -- new caps: video/x-raw, width=1280, height=720, format=(string)I420, framerate=30/1 NvMMLiteOpen : Block : BlockType = 4 ===== NvVideo: NVENC ===== NvMMLiteBlockCreate : Block : BlockType = 4 H264: Profile = 66 Level = 0 [gstreamer] gstreamer message latency ==> encoder [gstreamer] gstreamer stream status CREATE ==> src [gstreamer] gstreamer stream status ENTER ==> src NVMEDIA: Need to set EMC bandwidth : 376000 NvVideo: bBlitMode is set to TRUE [gstreamer] gstreamer message qos ==> encoder [gstreamer] gstreamer mysource ERROR Internal data stream error. [gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline1/GstAppSrc:mysource: streaming stopped, reason not-linked (-1)
When I try to create an rtsp stream using the nano_llm docker image I get a GStreamer internal data stream error. WebRTC works fine with the same video device.
This is the command I'm running:
Here's the full output of the command: