Camera streaming question

yury-sannikov commented 2 years ago

Hi @roflcoopter,

I'm playing with gstreamer in the viseron/camera/stream.py. It seems GStreamer is processing RTSP stream and uses hardware acceleration. However, I do not see any calls to the read or pool methods of the Stream class.

My config is pretty basic:


cameras:
  - name: Front door
    mqtt_name: viseron_front_door
    host: 192.168.5.100
    port: 554
    username: admin
    password: "123"
    path: /cam/realmonitor?channel=4&subtype=0
    codec: h264_nvmpi
    width: 1920
    height: 1080
    fps: 25
    pix_fmt: nv12
    logging:
      level: debug
    motion_detection:
      interval: 1
      trigger_detector: false
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.9
        - label: pottedplant
          confidence: 0.9

I didn't dig into it much, GStreamer eat up all the time, but maybe you can spot something that I didn't catch.

yury-sannikov commented 2 years ago


[2021-12-08 22:54:40] [INFO    ] [viseron                 ] - Initializing...
[2021-12-08 22:54:41] [INFO    ] [viseron                 ] - Initializing NVR threads
[2021-12-08 22:54:41] [DEBUG   ] [viseron.nvr.front_door  ] - Initializing NVR thread
[2021-12-08 22:54:41] [DEBUG   ] [viseron.camera.front_door] - Initializing camera Front door
[2021-12-08 22:54:43] [DEBUG   ] [viseron.camera.front_door] - Resolution: 1920x1080 @ 25 FPS
[2021-12-08 22:54:43] [DEBUG   ] [viseron.camera.front_door] - Starting decoder thread
[2021-12-08 22:54:43] [DEBUG   ] [viseron.camera.front_door] - Running decoder front_door.object_detection at 1s interval, every 25 frame(s)
[2021-12-08 22:54:43] [DEBUG   ] [viseron.camera.front_door] - Camera Front door initialized
[2021-12-08 22:54:43] [DEBUG   ] [viseron.motion.front_door] - Initializing motion detector background_subtractor
[2021-12-08 22:54:43] [DEBUG   ] [viseron.motion.front_door] - Starting decoder thread
[2021-12-08 22:54:43] [DEBUG   ] [viseron.motion.front_door] - Running decoder front_door.motion_detection at 1.0s interval, every 25.0 frame(s)
[2021-12-08 22:54:43] [DEBUG   ] [viseron.motion.front_door] - Motion detector initialized
[2021-12-08 22:54:43] [DEBUG   ] [viseron.nvr.front_door  ] - Starting camera
[2021-12-08 22:54:43] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 22:54:43] [DEBUG   ] [viseron.recorder.front_door] - Initializing ffmpeg recorder
[2021-12-08 22:54:43] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1
[2021-12-08 22:54:43] [DEBUG   ] [viseron.recorder.front_door] - Creating folder /segments/Front door
[2021-12-08 22:54:43] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1
[2021-12-08 22:54:43] [DEBUG   ] [viseron.nvr.front_door  ] - Waiting for first frame
[2021-12-08 22:54:43] [DEBUG   ] [viseron.nvr.front_door  ] - NVR thread initialized
[2021-12-08 22:54:43] [INFO    ] [viseron                 ] - Initialization complete
[2021-12-08 22:54:43] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.731: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.732: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.732: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.732: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.732: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:356): GStreamer-CRITICAL **: 22:54:45.732: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 22:54:45] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 22:54:47] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 22:54:47] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 

[2021-12-08 22:56:57] [ERROR   ] [viseron.camera.stream.front_door] - Frame read 3110400, expected 3110400
[2021-12-08 22:56:57] [DEBUG   ] [viseron.nvr.front_door  ] - First frame received
[2021-12-08 22:56:57] [INFO    ] [viseron.camera.front_door] - FFMPEG frame grabber stopped
[2021-12-08 22:56:57] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 22:56:57] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvi[2021-12-08 22:56:57] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1, message repeated 2 times
[2021-12-08 22:56:57] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:395): GStreamer-CRITICAL **: 22:56:58.208: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 22:56:58] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 22:57:00] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 22:57:00] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 
[2021-12-08 22:59:10] [ERROR   ] [viseron.camera.stream.front_door] - Frame read 3110400, expected 3110400
[2021-12-08 22:59:10] [DEBUG   ] [viseron.nvr.front_door.object] - Objects: []
[2021-12-08 22:59:11] [INFO    ] [viseron.camera.front_door] - FFMPEG frame grabber stopped
[2021-12-08 22:59:11] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvi[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1, message repeated 2 times
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:427): GStreamer-CRITICAL **: 22:59:11.828: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 22:59:11] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 
[2021-12-08 23:01:15] [ERROR   ] [viseron.camera.stream.front_door] - Frame read 3110400, expected 3110400
[2021-12-08 23:01:15] [DEBUG   ] [viseron.nvr.front_door.object] - Objects: []
[2021-12-08 23:01:15] [INFO    ] [viseron.camera.front_door] - FFMPEG frame grabber stopped
[2021-12-08 23:01:15] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 23:01:15] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvi[2021-12-08 23:01:15] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1, message repeated 2 times
[2021-12-08 23:01:15] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:455): GStreamer-CRITICAL **: 23:01:17.943: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 23:01:17] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 23:01:26] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 23:01:26] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 
[2021-12-08 23:03:45] [ERROR   ] [viseron.camera.stream.front_door] - Frame read 3110400, expected 3110400
[2021-12-08 23:03:45] [DEBUG   ] [viseron.nvr.front_door.object] - Objects: []
[2021-12-08 23:03:45] [INFO    ] [viseron.camera.front_door] - FFMPEG frame grabber stopped
[2021-12-08 23:03:45] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 23:03:45] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvi[2021-12-08 23:03:45] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1, message repeated 2 times
[2021-12-08 23:03:45] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:483): GStreamer-CRITICAL **: 23:03:47.911: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 23:03:47] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 23:03:54] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 23:03:54] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 
[2021-12-08 23:05:58] [ERROR   ] [viseron.camera.stream.front_door] - Frame read 3110400, expected 3110400
[2021-12-08 23:05:58] [DEBUG   ] [viseron.nvr.front_door.object] - Objects: []
[2021-12-08 23:05:58] [INFO    ] [viseron.camera.front_door] - FFMPEG frame grabber stopped
[2021-12-08 23:05:58] [DEBUG   ] [viseron.camera.front_door] - Starting capture thread
[2021-12-08 23:05:58] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvi[2021-12-08 23:05:58] [ERROR   ] [viseron.camera.stream.front_door] - FFMPEG decoder command: /usr/bin/gst-launch-1.0 rtspsrc location=rtsp://*****:*****@192.168.5.95:554/stream0 ! rtph264depay ! h264parse ! queue leaky=2 ! omxh264dec enable-low-outbuffer=1 ! nvvidconv ! video/x-raw(memory:NVMM),format=(string)NV12 ! fdsink fd=1, message repeated 2 times
[2021-12-08 23:05:58] [ERROR   ] [viseron.camera.stream.front_door] - Frame read
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_caps_is_empty: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_caps_truncate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_caps_fixate: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_caps_get_structure: assertion 'GST_IS_CAPS (caps)' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_structure_get_string: assertion 'structure != NULL' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - (gst-launch-1.0:511): GStreamer-CRITICAL **: 23:05:59.209: gst_mini_object_unref: assertion 'mini_object != NULL' failed
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteOpen : Block : BlockType = 261 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - NVMEDIA: Reading vendor.tegra.display-size : status: 6 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - NvMMLiteBlockCreate : Block : BlockType = 261 
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - Allocating new output: 1920x1088 (x 9), ThumbnailMode = 0
[2021-12-08 23:05:59] [ERROR   ] [viseron.camera.stream.front_door] - OPENMAX: HandleNewStreamFormat: 3605: Send OMX_EventPortSettingsChanged: nFrameWidth = 1920, nFrameHeight = 1080 ```

roflcoopter commented 2 years ago

I have not looked into gstreamer yet, however the abstraction in #306 allows for a separate integration for it.

But it looks like the gstreamer command gives one frame then exits. What happens if you exec into the container and run the command on the commandline? Does it run continously?

yury-sannikov commented 2 years ago

I'm slowly creeping forward. My main issue was related to the combination of the H.264 codec in the Anpviz camera (the pretty bad one). The camera hangs up pretty often and reboots after a while.

As of now, my findings are:

Camera: H.265+ 1080P at 5FPS
gstreamer using hardware decoding (at least I hope so. jtop shows NVDEC ant 716mzh)
figured out invalid frame issue (forgot the -q argument for gstreamer)
was able to write every other 20th frame at FrameDecoder.decode_frame() into the .jpg using decoded_frame_umat_rgb and validate that the JPG is okay
top shows ~20% CPU for python3 process
top shows ~18% CPU for gst-launch, although, the same command launched from Ubuntu console consumes almost nothing
gstreamer video/x-raw,format=(string)NV12 and video/x-raw(memory:NVMM),format=(string)NV12 both works fine from the console, however, adding (memory:NVMM) into the steam.py prevent gstreamer from streaming. Though I did not notice any performance differences.

Tested gst-launch-1.0 CPU usage inside Docker. I used both autovideosink and fdsink fd=1

CMD ["/usr/bin/gst-launch-1.0", \
  "-q", \
  "rtspsrc", "location=rtsp://admin:123456@192.168.5.95:554/stream0", "!", \
  "rtph265depay", "!", \
  "h265parse", "!", \
  "omxh265dec", "!", \
  "video/x-raw,format=(string)NV12", "!", \
  "autovideosink"]

COPY rootfs/ /
COPY viseron /src/viseron/

in both cases, CPU usage of the gst-launch-1.0 is ~3%. Not sure why the same command consumes 18% CPU launched as subprocess.Popen()

So far I was not able to make motion_detection work. Will be doing more digging through the code tomorrow.

yury-sannikov commented 2 years ago

I was able to make it work on local Jetson nano using gstreamer and PyGObject

I put together some of the files from my experiment, though it's pretty raw https://gist.github.com/yury-sannikov/a0b17e506c853a4a4b3ea492fe2d750e The main drawbacks are:

gstreamer restart not implemented / not working
gstreamer probe not implemented although feasble
write segments not implemented

Since Viseron v2 is in progress, I think, I will circle back to this when it will be ready.

roflcoopter commented 2 years ago

Great work!

Hope to finish of v2 soon

roflcoopter / viseron

Camera streaming question #308