FFMPEG is crashing the intel NUC due to memory stranded at 100%

[emile-bodin]

Discussed in https://github.com/blakeblackshear/frigate/discussions/11189

^{Originally posted by **emile-bodin** May 1, 2024} ### Describe the problem you are having i'm running home assistant supervised 2024.04.4 with the frigate addon 0.13.2 (full access) with protection mode off It is running on: - Intel NUC - 4 cores Intel(R) Core(TM) i5-4250U - 8gb ram - nvme SSD I have setup a nest camera with the google nest integration and passing the stream trough the internal go2rtc Now some minutes after starting frigate this is causing the intel nuc to go to 100% memory usage, and grinding the system to a halt.  I initially thought it might be the detection mode, but after disabling it via the frigate mqtt integration, it also went again to 100% memory usage  I really could use some insights here ### Version 0.13.2-6476f8a ### Frigate config file ```yaml # mqtt mqtt: host: 192.168.86.104 user: frigate password: nestcam # recordings record: enabled: true retain: days: 14 mode: all events: retain: default: 30 mode: motion # global ffmpeg ffmpeg: output_args: record: preset-record-generic-audio-aac hwaccel_args: preset-vaapi go2rtc: hass: config: "/config" streams: hass-achtertuin: - hass://supervisor?entity_id=camera.achtertuin # cameras cameras: hass-achtertuin: # <--- this will be changed to your actual camera later enabled: false ffmpeg: inputs: - path: rtsp://127.0.0.1:8554/hass-achtertuin?mp4 input_args: preset-rtsp-restream roles: - record # objects: # track: # - person # - dog # motion: # mask: # - 116,192,668,78,678,25,109,85 ``` ### Relevant log output ```shell 2024-05-01 11:45:47.040782776 [INFO] Preparing new go2rtc config... 2024-05-01 11:45:47.107233402 [INFO] Got IP address from supervisor: 192.168.86.104 2024-05-01 11:45:47.164995984 [INFO] Got WebRTC port from supervisor: 8555 2024-05-01 11:45:47.828836411 [WARN] Using go2rtc binary from '/config/go2rtc' instead of the embedded one 2024-05-01 11:45:47.828844682 [INFO] Starting go2rtc... 2024-05-01 11:45:47.933630642 13:45:47.933 INF go2rtc version 1.8.5 linux/amd64 2024-05-01 11:45:47.934529168 13:45:47.934 INF [api] listen addr=:1984 2024-05-01 11:45:47.934556813 13:45:47.934 INF [rtsp] listen addr=:8554 2024-05-01 11:45:47.934827403 13:45:47.934 INF [webrtc] listen addr=:8555 2024-05-01 11:45:56.952289021 [INFO] Starting go2rtc healthcheck service... 2024-05-01 11:45:46.950530894 [INFO] Preparing Frigate... 2024-05-01 11:45:47.041276553 [INFO] Starting Frigate... 2024-05-01 11:45:50.236242838 [2024-05-01 13:45:50] frigate.app INFO : Starting Frigate (0.13.2-6476f8a) 2024-05-01 11:45:50.236424460 [2024-05-01 13:45:50] frigate.app INFO : Creating directory: /tmp/cache 2024-05-01 11:45:53.548208913 [2024-05-01 13:45:53] peewee_migrate.logs INFO : Starting migrations 2024-05-01 11:45:53.570332978 [2024-05-01 13:45:53] peewee_migrate.logs INFO : There is nothing to migrate 2024-05-01 11:45:53.615846784 [2024-05-01 13:45:53] frigate.app INFO : Recording process started: 319 2024-05-01 11:45:53.619917331 [2024-05-01 13:45:53] frigate.app INFO : go2rtc process pid: 87 2024-05-01 11:45:53.646061683 [2024-05-01 13:45:53] detector.cpu INFO : Starting detection process: 329 2024-05-01 11:45:53.662438743 [2024-05-01 13:45:53] frigate.app INFO : Output process started: 331 2024-05-01 11:45:53.662921873 [2024-05-01 13:45:53] frigate.detectors WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes. 2024-05-01 11:45:53.701077474 [2024-05-01 13:45:53] frigate.app INFO : Camera processor not started for disabled camera hass-achtertuin 2024-05-01 11:45:53.701388612 [2024-05-01 13:45:53] frigate.app INFO : Capture process not started for disabled camera hass-achtertuin 2024-05-01 11:46:25.170393058 [2024-05-01 13:46:25] frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to initialize PMU! (Permission denied) 2024-05-01 11:46:25.170398810 2024-05-01 11:47:03.853786238 [2024-05-01 13:47:03] frigate.util.services ERROR : Unable to poll intel GPU stats: Failed to initialize PMU! (Permission denied) 2024-05-01 11:47:03.853792841 ``` ### Frigate stats ```json { "cameras": { "hass-achtertuin": { "audio_dBFS": 0, "audio_rms": 0, "camera_fps": 0, "capture_pid": null, "detection_enabled": 1, "detection_fps": 0, "ffmpeg_pid": 0, "pid": null, "process_fps": 0, "skipped_fps": 0 } }, "cpu_usages": { "1": { "cmdline": "/package/admin/s6/command/s6-svscan -d4 -- /run/service", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "15": { "cmdline": "s6-supervise s6-linux-init-shutdownd", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "16": { "cmdline": "/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "24": { "cmdline": "s6-supervise nginx-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "25": { "cmdline": "s6-supervise frigate-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "26": { "cmdline": "s6-supervise s6rc-oneshot-runner", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "27": { "cmdline": "s6-supervise s6rc-fdholder", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "28": { "cmdline": "s6-supervise go2rtc", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "29": { "cmdline": "s6-supervise nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "30": { "cmdline": "s6-supervise go2rtc-healthcheck", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "31": { "cmdline": "s6-supervise go2rtc-log", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "32": { "cmdline": "s6-supervise frigate", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "41": { "cmdline": "/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "42": { "cmdline": "/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "79": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "80": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "82": { "cmdline": "s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "87": { "cmdline": "/config/go2rtc -config=/dev/shm/go2rtc.yaml", "cpu": "0.0", "cpu_average": "0", "mem": "0.3" }, "108": { "cmdline": "bash ./run.user go2rtc-healthcheck", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" }, "110": { "cmdline": "python3 -u -m frigate", "cpu": "0.9", "cpu_average": "3", "mem": "2.3" }, "122": { "cmdline": "nginx: master process nginx", "cpu": "0.0", "cpu_average": "0", "mem": "0.2" }, "147": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "148": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "149": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "150": { "cmdline": "nginx: worker process", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "151": { "cmdline": "nginx: cache manager process", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "305": { "cmdline": "frigate.logger ", "cpu": "0.0", "cpu_average": "0", "mem": "1.3" }, "319": { "cmdline": "frigate.recording_manager", "cpu": "0.2", "cpu_average": "0", "mem": "1.4" }, "328": { "cmdline": "/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(41)", "cpu": "0.0", "cpu_average": "0", "mem": "0.1" }, "329": { "cmdline": "frigate.detector.cpu ", "cpu": "0.1", "cpu_average": "0", "mem": "1.5" }, "331": { "cmdline": "frigate.output ", "cpu": "0.2", "cpu_average": "0", "mem": "1.5" }, "334": { "cmdline": "ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:", "cpu": "0.0", "cpu_average": "0", "mem": "0.2" }, "335": { "cmdline": "ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:", "cpu": "0.0", "cpu_average": "0", "mem": "0.2" }, "441": { "cmdline": "sleep 30s", "cpu": "0.0", "cpu_average": "0", "mem": "0.0" } }, "detection_fps": 0, "detectors": { "cpu": { "detection_start": 0, "inference_speed": 10, "pid": 329 } }, "gpu_usages": { "error-gpu": { "gpu": -1, "mem": -1 } }, "processes": { "go2rtc": { "pid": 87 }, "logger": { "pid": 305 }, "recording": { "pid": 319 } }, "service": { "last_updated": 1714564074, "latest_version": "0.13.2", "storage": { "/dev/shm": { "free": 3936.3, "mount_type": "tmpfs", "total": 3936.8, "used": 0.5 }, "/media/frigate/clips": { "free": 65293.8, "mount_type": "ext4", "total": 111142.2, "used": 40157.6 }, "/media/frigate/recordings": { "free": 65293.8, "mount_type": "ext4", "total": 111142.2, "used": 40157.6 }, "/tmp/cache": { "free": 3936.8, "mount_type": "tmpfs", "total": 3936.8, "used": 0 } }, "temperatures": { }, "uptime": 121, "version": "0.13.2-6476f8a" } } ``` ### Operating system HassOS ### Install method HassOS Addon ### Coral version CPU (no coral) ### Any other information that may be helpful _No response_

Tried the latest go2rtc build, with supposably a refresh fix for google nest camera, but this did not help

[emile-bodin]

VAINFO OUTPUT
Return Code: 0

Process Output:

vainfo: VA-API version: 1.17 (libva 2.10.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Mobile - 2.4.1
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264MultiviewHigh : VAEntrypointVLD
VAProfileH264MultiviewHigh : VAEntrypointEncSlice
VAProfileH264StereoHigh : VAEntrypointVLD
VAProfileH264StereoHigh : VAEntrypointEncSlice
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
VAProfileJPEGBaseline : VAEntrypointVLD

FFPROBE OUTPUT
Stream 0:
Return Code: 0

Video:
Codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
Resolution: 1920x1080
FPS: 30/1

Audio:
Codec: Opus (Opus Interactive Audio Codec)

Stream INFO

{
  "hass-achtertuin": {
    "consumers": [
      {
        "medias": [
          "video, sendonly, ANY",
          "audio, sendonly, ANY"
        ],
        "remote_addr": "127.0.0.1:44644",
        "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.4\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 OPUS/48000/2\r\na=fmtp:97 minptime=10;useinbandfec=1\r\na=control:trackID=1\r\n",
        "send": 10583401,
        "senders": [
          "96 H264, bytes=10164466, receivers=1",
          "97 OPUS/48000/2, bytes=235142, receivers=1"
        ],
        "type": "RTSP passive consumer",
        "url": "rtsp://127.0.0.1:8554/hass-achtertuin?video&audio",
        "user_agent": "FFmpeg Frigate/0.13.2-6476f8a"
      }
    ],
    "producers": [
      {
        "medias": [
          "video, recvonly, H264",
          "video, sendonly, H264",
          "audio, recvonly, OPUS/48000/2",
          "audio, sendonly, OPUS/48000/2"
        ],
        "receivers": [
          "97 H264, bytes=10164466, senders=1",
          "101 OPUS/48000/2, bytes=235142, senders=1"
        ],
        "recv": 10787326,
        "remote_addr": "udp4 host 108.177.126.127:19305",
        "type": "Hass active producer",
        "url": ""
      }
    ]
  }
}