search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.21k stars 1.76k forks source link

[Support]: Output Recordings of h265 Streams to h264 Video #2444

Closed xCJPECKOVERx closed 2 years ago

xCJPECKOVERx commented 2 years ago

Describe the problem you are having

My camera is a Reolink 822a (4k, h265 rtsp stream). Recordings are saved as h265 mp4s and are therefore not viewable in most browsers.

Is there a set of output args for recording that will convert from h265 to h264, while only affecting performance when a recording is being made? I was able to get h264 recordings to work using the input and output args as described in the mjpeg cameras documentation, however it used a tonne of cpu resources even when not recording.

Any help or ideas would be appreciated.

Version

0.10.0-95BDF9F

Frigate config file

cameras:
  front_door:
    rtmp:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.0.80:554
          roles:
            - record
        - path: rtsp://user:password@192.168.0.80:554/h264Preview_01_sub 
          roles:
            - detect
    detect:
      width: 896
      height: 512
      fps: 4

Relevant log output

n/a

FFprobe output from your camera

Input #0, rtsp, from 'rtsp://user:password@192.168.0.80:554':
  Metadata:
    title           : Session streamed by "preview"
    comment         : 
  Duration: N/A, start: 0.003500, bitrate: N/A
    Stream #0:0: Video: hevc, yuv420p(tv), 3840x2160, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac, 16000 Hz, mono, fltp

Frigate stats

n/a

Operating system

UNRAID

Install method

Docker Compose

Coral version

M.2

Network connection

Wired

Camera make and model

Reolink 822a

Any other information that may be helpful

No response

blakeblackshear commented 2 years ago

Do you have the option to switch your cameras to H264? It's going to require an incredible amount of resources to re-encode to H264.

xCJPECKOVERx commented 2 years ago

Unfortunately I don't. I'm not sure if Reolink removed that ability or if it was just never available for their 4k cameras.

I did find another issue thread (about nvidia settings) where a user suggested the following record output args: -map 0:v -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v h264_nvenc -b:v 4M -r 15

This does successfully encode the recording into h264 with minimal impact on cpu (from <1% usage to 4% usage). The problem now is that if I change any of the above parameters then the streams fail; even the ones that don't have these output_args.

For example if I change the framerate to match the input (-r 25), the detect stream becomes very choppy and eventually fails, despite having the default input and output_args. The record stream also stops recording. Here are the errors:

[2021-12-14 10:09:03] frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures

[2021-12-14 10:09:03] frigate.video                  INFO    : front_door: ffmpeg process is not running. exiting capture thread...

[2021-12-14 10:09:13] watchdog.front_door            ERROR   : FFMPEG process crashed unexpectedly for front_door.

[2021-12-14 10:09:13] watchdog.front_door            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.

[2021-12-14 10:09:13] watchdog.front_door            ERROR   : You may have invalid args defined for this camera.

[2021-12-14 10:09:18] frigate.video                  INFO    : front_door: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures

[2021-12-14 10:09:18] frigate.video                  INFO    : front_door: ffmpeg process is not running. exiting capture thread...

[2021-12-14 10:09:23] watchdog.front_door            ERROR   : FFMPEG process crashed unexpectedly for front_door.

[2021-12-14 10:09:23] watchdog.front_door            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.

[2021-12-14 10:09:23] watchdog.front_door            ERROR   : You may have invalid args defined for this camera.

[2021-12-14 10:09:23] ffmpeg.front_door.detect       ERROR   : rtsp://user:pass@192.168.0.80:554/h264Preview_01_sub: Invalid data found when processing input

The h264Preview steam is the low res detect stream, not the 4k record stream with the output_args applied.

After the above error, ffmpeg spat out the following errors it had trying to handle the record stream:

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d758180] The cu_qp_delta 71 is outside the valid range [-26, 25].

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d789c40] Could not find ref with POC 24

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] The cu_qp_delta -27 is outside the valid range [-26, 25].

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Ignoring POC change between slices: 0 -> 24

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Non-matching NAL types of the VCL NALUs: 19 1

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Error parsing NAL unit #5.

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : Error while decoding stream #0:0: Invalid data found when processing input

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Could not find ref with POC 0

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d768a80] Non-matching NAL types of the VCL NALUs: 1 19

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d768a80] Error parsing NAL unit #1.

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7aadc0] Ignoring POC change between slices: 4 -> 17

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] Could not find ref with POC 17

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : Error while decoding stream #0:0: Invalid data found when processing input

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [rtsp @ 0x56541d652a40] RTP: PT=60: bad cseq 8615 expected=73db

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] The cu_qp_delta -95 is outside the valid range [-26, 25].

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] Ignoring POC change between slices: 0 -> 37

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] Non-matching NAL types of the VCL NALUs: 19 1

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] Error parsing NAL unit #4.

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7cbfc0] Could not find ref with POC 37

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d789c40] The cu_qp_delta 29 is outside the valid range [-26, 25].

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : Error while decoding stream #0:0: Invalid data found when processing input

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [rtsp @ 0x56541d652a40] RTP: PT=60: bad cseq 9b52 expected=8a92

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d768a80] The cu_qp_delta -31 is outside the valid range [-26, 25].

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d768a80] Non-matching NAL types of the VCL NALUs: 1 19

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d768a80] Error parsing NAL unit #1.

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Could not find ref with POC 0

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7bb6c0] Ignoring POC change between slices: 5 -> 9

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d7cbfc0] Could not find ref with POC 9

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : Error while decoding stream #0:0: Invalid data found when processing input

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : More than 1000 frames duplicated

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [rtsp @ 0x56541d652a40] RTP: PT=60: bad cseq c7bc expected=af32

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d779340] Ignoring POC change between slices: 3 -> 37

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   :     Last message repeated 2 times

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d789c40] Could not find ref with POC 37

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [rtsp @ 0x56541d652a40] RTP: PT=60: bad cseq d674 expected=c91e

[2021-12-14 10:09:23] ffmpeg.front_door.record       ERROR   : [hevc @ 0x56541d747b40] The cu_qp_delta -34 is outside the valid range [-26, 25].

Any idea what to make of all this? I wouldn't have expected changing output args on the record stream to cause the detect stream to also fail.

blakeblackshear commented 2 years ago

Please post your updated config.

xCJPECKOVERx commented 2 years ago

Here's all I have in the config file (by the way I am using the nvidia build, following the docs for docker config):

mqtt:
  host: 192.168.0.150
  user: user
  password: pass

#detectors
detectors:
  coral:
    type: edgetpu
    device: pci:0

objects:
  track:
    - person
    - car

#live streams
live:
  quality: 1

#birdseye
birdseye:
  quality: 1
  mode: continuous

#snapshots
snapshots:
  enabled: true
  timestamp: false
  bounding_box: false
  crop: false
  quality: 100
  retain:
    default: 3
    objects:
      person: 7

#record
record:
  enabled: true
  events:
    pre_capture: 10
    post_capture: 10
    objects:
      - car
      - person
    retain:
      default: 3
      objects:
        person: 7

ffmpeg:
  global_args: -hide_banner -loglevel warning
  hwaccel_args: []
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1
  output_args:
    detect: -f rawvideo -pix_fmt yuv420p
    record: -map 0:v -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v h264_nvenc -b:v 4M -r 25
    rtmp: -c copy -f flv

#cameras
cameras:
  front_door:
    rtmp:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://user:pass@192.168.0.80:554
          roles:
            - record
        - path: rtsp://user:pass@192.168.0.80:554/h264Preview_01_sub
          roles:
            - detect
    detect:
      width: 640
      height: 360
      fps: 4

Currently the record stream fails as well as the detect stream. If the record framerate output arg is changed to 15, everything works fine. Are the stream inputs designed to fail all if one of them has issues?

blakeblackshear commented 2 years ago

They are running as separate ffmpeg processes, so not sure why that would cause both to fail.

xCJPECKOVERx commented 2 years ago

Weird. Well I guess it doesn't break everything, as I can still download recordings or view h265 recordings directly instead of encoding to h264. Thanks.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

a2nt commented 2 years ago

If someone seeks for some low-end solution:

# lspci            
00:00.0 Host bridge: Intel Corporation Gemini Lake Host Bridge (rev 06)
00:02.0 VGA compatible controller: Intel Corporation GeminiLake [UHD Graphics 600] (rev 06)
# lscpu
  Model name:            Intel(R) Celeron(R) J4125 CPU @ 2.00GHz

cat ./config.yml

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1
  output_args:
      record: -map 0:v -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v h264_qsv -b:v 4M -r 5