[Support]: Problem when trying to restream with a single connection open to the camera

[cpadil]

Describe the problem you are having

Is important to note that after upgrading to v12 everything was working fine.

The problem started when I tried to enable go2rtc with a single connection to the camera. It actually works but just for ~24 hours, after that the stream to frigate gets interrupted and never gets back. However the go2rtc live view (webrtc & mse) STILL works. Seems that go2rtc is still working but frigate is not able to recover the connection.

In go2rtc logs there is a timeout warning. In frigate logs, it says that the camera exceeded fps limit, as per the system view, the camera is at ~100 fps which is imposible, the camera is a Wyze V3 with the rtsp firmware and is not capable of more than 20 fps.

The workaround is to restart frigate container, but the same issue happens after ~24 hours.

Installation: Proxmox -> LXC (Debian) -> Docker 2 cores 4 GB RAM i5 11600 Coral TPU M.2

Version

0.12.0-DA3E197

Frigate config file

mqtt:
  host: XXX
  user: mqtt-user
  password: XXX

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264

snapshots:
  timestamp: True
  enabled: True
  retain:
    default: 1

timestamp_style:
  position: "tl"
  format: "%Y-%m-%d %H:%M:%S"

#new go2rtc changes
go2rtc:
  streams:
    bed_cam: 
      - rtsp://wyzecam:XXX@XXX/live
      - "ffmpeg:bed_cam"
    main_cam: 
      - rtsp://wyzecam:XXX@XXX/live
      - "ffmpeg:main_cam"
  webrtc:
        candidates:
              - 10.26.120.2:8555
              - stun:8555

cameras:
  bed_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/bed_cam #new go2rtc changes
          input_args: preset-rtsp-restream #new go2rtc changes
          roles:
            - detect
    objects:
      track:
        - person
      mask: 1000,365,1377,417,1399,164,1023,82
      filters:
        person:
          min_area: 295000
    zones:
      office:
        coordinates: 1410,1080,1195,801,1082,421,1035,359,1002,275,753,252,424,227,0,262,0,1080
    motion:
      mask: 1000,365,1377,417,1399,164,1023,82
  main_cam:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/main_cam #new go2rtc changes
          input_args: preset-rtsp-restream  #new go2rtc changes
          roles:
            - detect
    objects:
      track:
        - person
        - cat
      filters:
        person:
          min_score: 0.6
          min_area: 110000
          threshold: 0.75
        cat:
          min_score: 0.5
          threshold: 0.75
          mask:
            - 590,272,894,269,862,604,858,693,957,844,1319,859,1306,652,1616,657,1810,272,1920,266,1920,1080,610,1080,373,932,452,667,375,449

detect:
  width: 1920
  height: 1080
  fps: 20

detectors:
  coral1:
    type: edgetpu
    device: pci

Relevant log output

--frigate--
2023-04-14 18:07:04.853595620  [2023-04-14 18:07:04] watchdog.bed_cam               INFO    : bed_cam exceeded fps limit. Exiting ffmpeg...
2023-04-14 18:07:04.853597993  [2023-04-14 18:07:04] watchdog.bed_cam               INFO    : Waiting for ffmpeg to exit gracefully...

--go2rtc--
2023-04-14 18:06:56.849469150  18:06:56.849 WRN github.com/AlexxIT/go2rtc/cmd/streams/producer.go:133 > error="read tcp 10.26.120.2:33528->10.26.140.3:554: i/o timeout" url=rtsp://wyzecam:XXX@XXX/live

FFprobe output from your camera

[rtsp @ 0x55e8bab53980] UDP timeout, retrying with TCP
Input #0, rtsp, from 'rtsp://wyzecam:XXX@XXX/live':
  Metadata:
    title           : Session streamed by the WYZE Media Server
    comment         : live
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 20 fps, 20 tbr, 90k tbn, 40 tbc
    Stream #0:1: Audio: pcm_alaw, 16000 Hz, 1 channels, s16, 128 kb/s

Frigate stats

coral1
P-ID    Inference Speed CPU %   Memory %
554 7.7 ms  0.0%    0.1%

intel-qsv
GPU %   Memory %
1.37 %  - %

Bed Cam
FFPROBE
Process P-ID    FPS CPU %   Memory %
ffmpeg  30177   101.7   - % - %
Capture 574 0.3 0.0%    0.0%
Detect  561 disabled    0.0%    0.1%

Operating system

Debian

Install method

Docker Compose

Coral version

M.2

Network connection

Wireless

Camera make and model

Wyze V3

Any other information that may be helpful

No response

[NickM-27]

go2rtc doesn't work well with cheap / low quality cameras / streams like wyze, I wouldn't recommend running it that way

[NickM-27]

by the way, adding - "ffmpeg:bed_cam" to your go2rtc config without any modifiers does nothing

[cpadil]

go2rtc doesn't work well with cheap / low quality cameras / streams like wyze, I wouldn't recommend running it that way

Got it, so this means this an issue with go2rtc? Even though the original stream is still visible in go2rtc modes (me &webrc)

[NickM-27]

rtsp and mse / webrtc are different streams with different logic / technologies. This is also a common issue seen in the beta with cheap cameras like the wyze and similar

[my-umd]

same/similar to my issue #5976? Looks the container didn't crash in this issue though.

[cpadil]

same/similar to my issue #5976? Looks the container didn't crash in this issue though.

indeed, looks like a similar issue, but as you mentioned, in my case the container is not stopped... have you tried removing the input args? I'm not sure what is the difference with "preset-rtsp-generic" (default when nothing is mentioned) and "preset-rtsp-restream" which is mentioned in the documentation for a single connection to the camera

I'll test that and see if is more stable

[my-umd]

same/similar to my issue #5976? Looks the container didn't crash in this issue though.

have you tried removing the input args? I'm not sure what is the difference with "preset-rtsp-generic" (default when nothing is mentioned) and "preset-rtsp-restream" which is mentioned in the documentation for a single connection to the camera

I didn't try. I remember I tried keeping go2rtc settings in the config but directly connecting to camera streams without using any go2rtc restreaming. My container still crashed. Using the same config file but removing go2rtc settings section, my Frigate docker runs just fine (This is how I am running now).

I bet there are people who do not use Frigate but use go2rtc directly to restream Wyze cam. But I just could not find any report that has the same issue as mine. Looks to me that the issue is with go2rtc plus Frigate plus Wyze plus my platform. I am thinking to install a go2rtc docker separately to restream my Wyze to see if it crashes.

BTW, also worth noting is that seems I had an out of memory issue that is not reported in your case. I have 6 cameras whereas you have 2. Not sure if you will eventually run out of memory if you let your system run longer.

[my-umd]

As a side note, your detect fps seems too high. I remember it should be under 5.

[NickM-27]

The default is 5, typically that's enough but for fast objects like cars maybe 10 would be needed. Definitely not 20.

[cpadil]

I can confirm that Frigate is more stable with the generic preset for RTSP using a single connection for the detection and live view (with go2rtc).

36+ hours without the error reported above.

Since Wyze V3 RTSP firmware does not allow to set the FPS, I've to set the fps to 20 (as per Wyze docs). If I don't set it to that I constantly get errors like "exceeded fps limit. Exiting ffmpeg..."

I'll try to modify this with the "wz_mini_hacks" since I do notice a relevant reduction in CPU usage with less fps.

[NickM-27]

For most users using good cameras, using the generic preset with the single stream method caused hitching in the recordings, just FYI

[my-umd]

I can confirm that Frigate is more stable with the generic preset for RTSP using a single connection for the detection and live view (with go2rtc).

36+ hours without the error reported above.

Thanks for the update. Then looks this issue is not the same as mine (#5976). Mine crashes with go2rtc setting section in the config, even without actually using it (i.e., I still directly connect to cameras, not using any restreaming).

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.