search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.22k stars 1.76k forks source link

[Config Support]: Strange broken image #8905

Closed sebastian-tischer closed 9 months ago

sebastian-tischer commented 11 months ago

Describe the problem you are having

Hey team,

I finally moved to unraid on HPE Microserver Gen 10plus v2 and solved a lot of my problems.

Unfortunately, I am having broken images, quite constantly - no idea why 🤣 I bet you have an idea and can help me.

Bildschirmfoto 2023-12-09 um 17 24 58

Thank you and warm regards Sebastian

Version

0.12.1-367D724

Frigate config file

mqtt:
  enabled: true
  host: 192.168.2.163
  port: 1883
  user: hass-mqtt
  password: mqtt

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-nvidia-h264

motion:
  threshold: 15
  improve_contrast: True

detect:
  enabled: True
  width: 1280
  height: 720
  #width: 640
  #height: 360
  fps: 5

objects:
  filters:
    person:
      min_score: 0.7
      threshold: 0.7
  track:
    - person

birdseye:
  enabled: True
  mode: continuous
#      restream: true

record:
  enabled: True
  events:
    pre_capture: 60
    post_capture: 60
    objects:
      - person
    retain:
      default: 5
      mode: motion
      objects:
        person: 15

snapshots:
  enabled: True
  clean_copy: True
  bounding_box: True
  crop: False
  retain:
    default: 5
    objects:
      person: 15

cameras:
  Einfahrt:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Einfahrt
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Einfahrt-Detect
          input_args: preset-rtsp-restream
          roles:
            - detect
    motion:
      mask:
#        - 108,131,140,126,129,75,95,83
        - 176,221,184,272,285,266,278,219,268,154,177,169
    objects:
      filters:
        person:
          mask:
#            - 106,132,138,128,130,84,96,87 #GeringeAuflösung
            - 189,224,189,271,228,270,286,270,276,157,186,163
  Eingang:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Eingang
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Eingang-Detect
          input_args: preset-rtsp-restream
          roles:
            - detect
  Garten:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Garten
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Garten-Detect
          input_args: preset-rtsp-restream
          roles:
            - detect
    motion:
      mask:
#        - 201,71,0,106,0,54,198,28 #GeringeAuflösung
        - 0,216,0,107,179,79,394,55,401,148
    objects:
      filters:
        person:
          mask:
#            - 0,108,204,74,202,21,0,56 #GeringeAuflösung
            - 399,137,0,210,0,98,393,45
  Garage:
    ffmpeg: 
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Garage
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Garage-Detect
          input_args: preset-rtsp-restream
          roles:
            - detect
#      Wohnzimmer:
#        ffmpeg:
#          output_args:
#            record: preset-record-generic-audio-copy
#          inputs:
#            - path: rtsp://127.0.0.1:8554/Wohnzimmer
#              input_args: preset-rtsp-restream
#              roles:
#                - record
#            - path: rtsp://127.0.0.1:8554/Wohnzimmer-Detect
#              input_args: preset-rtsp-restream
#              roles:
#                - detect
go2rtc:
  streams:
    Einfahrt:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.141/livestream/12
#      - "ffmpeg:Einfahrt#video=h264#audio=aac"
    Einfahrt-Detect:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.141/livestream/11
#      - "ffmpeg:Einfahrt-Detect#video=h264#audio=aac"
    Eingang:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.127/livestream/12
#      - "ffmpeg:Eingang#video=h264#audio=aac"
    Eingang-Detect:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.127/livestream/11
#      - "ffmpeg:Eingang-Detect#video=h264#audio=aac"
    Garten:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.128/livestream/12
#      - "ffmpeg:Garten#video=h264#audio=aac"
    Garten-Detect:
      - rtsp://operator-kamera:qyw8xU8mkr.zh4R7znL3@192.168.2.128/livestream/11
#      - "ffmpeg:Garten-Detect#video=h264#audio=aac"
    Garage:
      - rtsp://operator-kamera:McN9foGXqtK2xPi@192.168.2.147:554/stream1
#      - "ffmpeg:Garage#video=h264#audio=aac"
    Garage-Detect:
      - rtsp://operator-kamera:McN9foGXqtK2xPi@192.168.2.147:554/stream1
#      - "ffmpeg:Garage-Detect#video=h264#audio=aac"
#    Wohnzimmer:
#      - rtsp://operator-kamera:McN9foGXqtK2xPi@192.168.2.148:554/stream1
#      - "ffmpeg:Wohnzimmer#video=h264#audio=aac"
#    Wohnzimmer-Detect:
#      - rtsp://operator-kamera:McN9foGXqtK2xPi@192.168.2.148:554/stream1
#      - "ffmpeg:Wohnzimmer-Detect#video=h264#audio=aac"

Relevant log output

FFProbe:
[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":1080,"width":1920},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":1440,"width":2560},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}}]

Nothing major in the log, except that:

2023-12-09 17:29:37.544370825  [2023-12-09 17:29:37] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Eingang. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:37.544702854  [2023-12-09 17:29:37] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:37.544853644  [2023-12-09 17:29:37] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garten. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:47.545089696  [2023-12-09 17:29:47] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Einfahrt. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:47.545849856  [2023-12-09 17:29:47] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Eingang. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:47.545951307  [2023-12-09 17:29:47] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:47.545996202  [2023-12-09 17:29:47] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garten. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:57.544917745  [2023-12-09 17:29:57] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Einfahrt. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:57.544921331  [2023-12-09 17:29:57] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Eingang. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:57.546097654  [2023-12-09 17:29:57] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:29:57.546100568  [2023-12-09 17:29:57] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garten. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:07.546176882  [2023-12-09 17:30:07] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Einfahrt. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:07.546260408  [2023-12-09 17:30:07] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Eingang. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:07.547056331  [2023-12-09 17:30:07] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:07.547251483  [2023-12-09 17:30:07] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garten. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:17.546461314  [2023-12-09 17:30:17] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Einfahrt. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:17.546965106  [2023-12-09 17:30:17] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Eingang. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:17.547460835  [2023-12-09 17:30:17] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage. Keeping the 5 most recent segments out of 6 and discarding the rest...
2023-12-09 17:30:17.547799709  [2023-12-09 17:30:17] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garten. Keeping the 5 most recent segments out of 6 and discarding the rest...

Frigate stats

DETECTORS
coral
P-ID    Inference Speed CPU %   Memory %
290 8.35 ms 9.3%    0.4%

GPU
NVIDIA T400 4GB
GPU %   Memory %
6 % 26.8 %

Operating system

UNRAID

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

NickM-27 commented 11 months ago

This is usually a camera issue that occurs while decoding. Maybe try a different profile on the camera settings

sebastian-tischer commented 11 months ago

As always thanks for the fast reply and your help :) I switched from h.264 to h.265 hevc profile on the cam and also higher compression - looks good so far.

Only question - do I need to change something in my config?

Furthermore - in your docu I only see this: hwaccel_args: preset-nvidia-h264 Can I also use => hwaccel_args: preset-nvidia-h265

sebastian-tischer commented 11 months ago

Just found it, I used hwaccel_args: preset-nvidia-h265 with the high-profile h.265 hevc in the camera.

What else in the camera profile can cause this, saw it 1-2 times since the switch :(

sebastian-tischer commented 10 months ago

Hi @NickM-27 I checked the last weeks and can't get rid of the image errors.

I tried to check the rtsp stream via VLC and it arrives great without any issues. Only in Frigate the issue comes up.

Profile in camera is h265: HEVC and 1280 kbps.

Anything I can check?! :)

NickM-27 commented 10 months ago

you could try connecting with udp instead of tcp, this is what vlc does

sebastian-tischer commented 10 months ago

Thanks, I just did it that way:

cameras:
  Einfahrt:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Einfahrt
          input_args: preset-rtsp-restream preset-rtsp-udp
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/Einfahrt-Detect
          input_args: preset-rtsp-restream preset-rtsp-udp
          roles:
            - detect
    motion:
      mask:
#        - 108,131,140,126,129,75,95,83
        - 176,221,184,272,285,266,278,219,268,154,177,169
    objects:
      filters:
        person:
          mask:
#            - 106,132,138,128,130,84,96,87 #GeringeAuflösung
            - 189,224,189,271,228,270,286,270,276,157,186,163

Strangely, I get a stream with mse in camera single view, but not in the overview.

Bildschirmfoto 2023-12-29 um 20 00 28 Bildschirmfoto 2023-12-29 um 20 00 16
NickM-27 commented 10 months ago

oh you can't connect to the restream via udp, you need to connect to the camera using udp. This requires using ffmpeg in go2rtc, not 100% sure which input is used for udp but it will be in the go2rtc docs

sebastian-tischer commented 10 months ago

I am not sure, but I might be one of the dumbest on earth... :D

I set the detect resolution to 1080p:

detect:
  enabled: True
  width: 1280
  height: 720

But then I wrongly changed the detect and record URL (detect was 1440p and record was 1080p).

Might this be the issue - as of right now it looks good :)

sebastian-tischer commented 10 months ago

Not there yet, but better 👍

I now adjusted the go2rtc config for udp and will check - for your reference maybe :)

go2rtc:
  streams:
    Einfahrt:
      - rtsp://xx:xx@192.168.x.xxx/livestream/11#input=rtsp/udp
sebastian-tischer commented 10 months ago

Wasn't the solution :(

In VLC I just enter the rtsp URL as above (without the #input...) and no issues at all. In Frigate I get these strange image errors :(

Any other idea I could try?

NickM-27 commented 10 months ago

these types of issues are common with reolink cameras on rtsp, it is possible that vlc is doing something to handle the packet loss or just dropping frames that contain corruption

sebastian-tischer commented 10 months ago

Unfortunately I don't have reolink 🤣 I have Instar from a German company.

I checked in the webinterface of the cam, the app of the cam, vlc => nothing like that :(

sebastian-tischer commented 10 months ago

I might need to switch to webrtc or what do you think?

https://docs.frigate.video/configuration/live#webrtc-extra-configuration

NickM-27 commented 10 months ago

ah gotcha, in any case it seems VLC may be doing something, as the image corruption is characteristic of packet loss / incorrect h.26X encoding

I might need to switch to webrtc or what do you think?

https://docs.frigate.video/configuration/live#webrtc-extra-configuration

that would have nothing to do with the decoded stream having artifacts

sebastian-tischer commented 10 months ago

I changed it back to tcp (since it is better for cooping with packet loss, as I understand). In the camera, I deactivated the variable bitrate and increased compression. I will check, but looks good and seems to be a camera topic - I will write you latest tomorrow, if it fixed it :)

sebastian-tischer commented 10 months ago

To bomb you further with messages here - that was not the solution 🤣

I now changed from H.265 to H.264 and this seems to work - as of now... keep you updated.

github-actions[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.