search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18k stars 1.64k forks source link

[Config Support]: Live stream laggy - in HA frigate or dashboard cards - not so in VLC (rtsp) - reolink doorbell #7589

Closed jcat59 closed 1 year ago

jcat59 commented 1 year ago

Describe the problem you are having

I have frigate HAOS addon set up - and having an issue with laggy live view within home assistant. This whether I view with frigate live card, or other camera entity. Is there anything I can do to help this? I'll have a USB coral tomorrow, but I'm guessing that won't help this issue.

Prior to installing Frigate, I had used RTSPtoWebRTC HAOS add on to display the live stream in HA, and that did not lag. I have removed that add on.

I am using HAOS baremetal, with frigate add on - with intel i5 processor (HP 600 G2, 8gb ram, SSD drives) - camera is a reolink wifi doorbell. Everything else is working other than the laggy video - just installed the past 2 days.

my reolink wifi doorbell camera specs Main stream - 2560/1920, FPS (20), 4096 max bitrate, 1x interframe Sub stream - 640/480, FPS (10), 256 max bitrate, 4x interframe

Version

latest

Frigate config file

environment_vars: #added as some have trouble with default for old intel (like current machine)
  LIBVA_DRIVER_NAME: i965
mqtt:
  host: 192.168.86.45
  user: user
  password: password

detectors:
  cpu:  #CPU inference at around 80ms
    type: cpu
    num_threads: 2
  #coral:   #Coral inference ? Comment out CPU once I have the coral
    #type: edgetpu
    #device: usb

cameras:
  frontdoor:
    ffmpeg:
      hwaccel_args: preset-vaapi     
      inputs:
      - path: rtsp://user:password@192.168.86.40:554/h264Preview_01_main
        roles:
          - record
      - path: rtsp://user:password@192.168.86.40:554/h264Preview_01_sub
        roles:
          - detect #detecting with sub stream, which is fluent (lower res)
    record:
      enabled: true
      events:
        required_zones:
          - front_door
    snapshots:
      enabled: true
      required_zones:
        - front_door
    detect: #height and width of reolink sub stream, which is fluent for detection
      enabled: true    
      width: 640
      height: 480
      fps: 5
    objects:
      track:
       - person
    motion:
      mask: #stops motion from being detected in this area, better performance
       - 429,0,640,79,312,232,0,133,0,38,0,0 #mask of date/time stamp and skyline
       #- 248,347,326,545,571,473,778,484,782,301,549,297 #2nd mask if needed
    rtmp:
      enabled: false
    zones:
      front_door:  #maps out the zone so no events/snapshots are created unless they step into my area
        coordinates: 419,480,351,443,352,369,270,355,166,427,81,392,72,274,0,285,0,480,160,480
        objects:
          - person

Relevant log output

none

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Any other information that may be helpful

I am using HAOS baremetal, with frigate add on - with intel i5 processor (HP 600 G2, 8gb ram, SSD drives) - camera is a reolink wifi doorbell. Everything else is working other than the laggy video - just installed the past 2 days.

my reolink wifi doorbell camera specs Main stream - 2560/1920, FPS (20), 4096 max bitrate, 1x interframe Sub stream - 640/480, FPS (10), 256 max bitrate, 4x interframe

NickM-27 commented 1 year ago

I'd suggest using the reolink configuration recommended in the docs https://docs.frigate.video/configuration/camera_specific#reolink-cameras

jcat59 commented 1 year ago

I'd suggest using the reolink configuration recommended in the docs https://docs.frigate.video/configuration/camera_specific#reolink-cameras

Thanks @NickM-27 , unfortunately I tried this and the performance is exactly the same - see relevant change below. Considering that, it seems better to leave as I had originally?

go2rtc: streams: reolink_doorbell_main:

NickM-27 commented 1 year ago

The problem is your naming of the streams, none of the stream names match the name of the camera

jcat59 commented 1 year ago

The problem is your naming of the streams, none of the stream names match the name of the camera

@NickM-27 - I made a change per your feedback, unfortunately with the same results after all said and done. Still a couple seconds lag. Details below.

I updated the camera name in Reolink admin to 'ReolinkDoorbell', then I rebooted and updated my main stream to 'ReolinkDoorbell' and sub stream to 'ReolinkDoorbell_sub'. I also had Frigate crash after this HTTP change due to an error with timestamps - and I added the below items per an issue I found with the same crash - https://github.com/blakeblackshear/frigate/issues/4131

hwaccel_args:

jcat59 commented 1 year ago

It crashed again - going to try and revert back to original code.

2023-08-27 20:35:40.704176324 [2023-08-27 16:35:40] ffmpeg.frontdoor.record ERROR : [segment @ 0x55ea939af380] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

NickM-27 commented 1 year ago

You shouldn't use manual args, at this point I'm not sure the issue, my reolink doorbell works flawlessly with the recommended config. What method of stream are you using in the frigate webui?

jcat59 commented 1 year ago

You shouldn't use manual args, at this point I'm not sure the issue, my reolink doorbell works flawlessly with the recommended config. What method of stream are you using in the frigate webui?

When I look in the webui, it shows JSMPEG - I don't see any option to change that... I did notice I have audio now with the recommended config

Do you have the time stamp on for your camera as well? Not sure why I'm getting that error. Would you mind sharing your config, or is it pretty much like mine with the recommended config (minus manual args)?

jcat59 commented 1 year ago

I wonder if it would help for me to install go2rtc add on. I originally tried that, but ended up getting confused on whether I actually even need the add on - since it appears to be already running with Frigate. It also starts getting into the restream topic....

Per below it sounds like you can get lower latency with go2rtc?

From Frigate docs

Use of the bundled go2rtc is optional. You can still configure FFmpeg to connect directly to your cameras. However, adding go2rtc to your configuration is required for the following features:

WebRTC or MSE for live viewing with higher resolutions and frame rates than the jsmpeg stream which is limited to the detect stream Live stream support for cameras in Home Assistant Integration RTSP (instead of RTMP) relay for use with other consumers to reduce the number of connections to your camera streams

NickM-27 commented 1 year ago

If jsmpeg is the only option then the stream and camera names are not matching correctly.

Using external go2rtc, the live stream features are not supported inside frigate or frigate card.

jcat59 commented 1 year ago

If jsmpeg is the only option then the stream and camera names are not matching correctly.

Using external go2rtc, the live stream features are not supported inside frigate or frigate card.

I'm not sure what you mean by the stream and camera names not matching. Within Reolink settings, the camera is named 'ReolinkDoorbell'. I have 2 streams in my current config, one named 'ReolinkDoorbell' and one named 'ReolinkDoorbell_sub'. Am I missing something? Is there somewhere else I need to update?

NickM-27 commented 1 year ago

The reolink settings don't matter at all. In go2rtc the name does not match frontdoor thus frigate doesn't know what camera it's related to.

jcat59 commented 1 year ago

The reolink settings don't matter at all. In go2rtc the name does not match frontdoor thus frigate doesn't know what camera it's related to.

Ok got it - I made that change and rebooted. Now - the live stream is super smooth, however, there is a ~15 second delay before you see whatever happened. Even if I shut off detect/motion/events. Is that how it's supposed to be, or is there one more tweak I can make :)? Before I made this change it was pretty much immediate, just super choppy.

NickM-27 commented 1 year ago

Which type of stream is being used?

jcat59 commented 1 year ago

When you ask what type of stream - do you mean what I've selected in the frigate UI (webrtc), or what I have in the config? Does the selection in the UI drive what you see in the frigate card?

What I've noticed - is that the frigate live card on my mobile app is always at least 15 seconds behind. It shows for example 06:53:01, but that same card on my PC shows 06:53:17, which is also what the frigate UI shows. So the PC is current, but the mobile (iPhone) is always behind. No matter what I do to refresh the mobile, it remains behind. I noticed the card on the PC has a little bar and timer, where you can drag it to catch up if it falls behind. No such thing for the same card on the mobile. Thoughts?

Then while looking further it crashed again with this error :(

2023-08-27 23:03:06.418580159 [2023-08-27 19:03:06] ffmpeg.ReolinkDoorbell.record ERROR : [segment @ 0x5637ea96e380] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

NickM-27 commented 1 year ago

It's better to test with frigate dashboard first. You need to configure the frigate card to use go2rtc as the stream provider. By default it is HLS which has the delay.

jcat59 commented 1 year ago

Thanks so much @NickM-27 - performance is great now! I'm good with this topic and will close out. I'll probably have to open another topic for the crashes if that continues