search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19k stars 1.73k forks source link

[Support]: hwaccel stopped working on raspberry pi after 5.15.61 kernel update #3780

Closed kylehendricks closed 1 year ago

kylehendricks commented 2 years ago

Describe the problem you are having

Everything was working fine with my frigate setup. I've been using the 0.11 betas/rc's since the beginning. All of the sudden one of my cameras stopped working in frigate completely. Blue Iris was still working fine with it. VLC opened the configured URL no problem. I've issues once in a while in the past where I'd see events stop happening so I'd just bounce frigate and all would be well. This time, bouncing frigate and the rebooting the camera hasn't helped.

The really strange thing is that I can't think of anything that has changed. IIRC, the camera was working fine with 0.11RC2 for at least a few days before this randomly started happening. No matter what I tried I just get repeating No frames received from driveway in 20 seconds errors. I do have another camera that is the same brand but a little bit older model that continued to work fine. I removed it from the config since the issue with the one camera still occurs.

Version

0.11.0-c461c9e

Frigate config file

mqtt:
  host: mqtt.home
  user: 'XXXX'
  password: 'XXXXXXX'

database:
  path: /db/frigate.db

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args:
    - -c:v
    - h264_v4l2m2m

record:
  enabled: True
  events:
    retain:
      default: 30

snapshots:
  enabled: True
  retain:
    default: 30

rtmp:
  enabled: False

cameras:
  driveway:
    ffmpeg:
      inputs:
        - path: rtsp://XXXXX:XXXXXX@driveway.cam.home:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - rtmp
            - record
        - path: rtsp://XXXXXX:XXXXXXX@driveway.cam.home:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
    detect:
      width: 1280
      height: 720
      fps: 5
    motion:
      mask:
        - 1230,441,1252,310,1259,120,1257,0,1280,0,1280,720,1130,720
        - 396,116,649,67,824,57,1002,70,1134,94,1259,122,1257,0,949,0,819,0,626,0,0,0,0,260
    zones:
      driveway_entry:
        coordinates: 819,248,639,209,1152,99,1202,124
      driveway_main:
        coordinates: 1095,720,332,720,244,560,174,308,425,217,476,233,528,225,630,214,821,266,867,305,1207,433
    objects:
      track:
        - person
        - car
        - dog

Relevant log output

frigate  | [2022-09-07 02:01:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.                                                       
frigate  | [2022-09-07 02:01:17] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...                                         
frigate  | [2022-09-07 02:01:27] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.                                                          
frigate  | [2022-09-07 02:01:27] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.                                        
frigate  | [2022-09-07 02:01:45] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:53484]                                
frigate  | [2022-09-07 02:01:47] watchdog.driveway              INFO    : No frames received from driveway in 20 seconds. Exiting ffmpeg...                                          
frigate  | [2022-09-07 02:01:47] watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...                                                                   
frigate  | [2022-09-07 02:01:50] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51066]                                   
frigate  | [2022-09-07 02:02:17] watchdog.driveway              INFO    : FFmpeg didnt exit. Force killing...
frigate  | [2022-09-07 02:02:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:02:17] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
frigate  | [2022-09-07 02:02:27] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.
frigate  | [2022-09-07 02:02:27] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-09-07 02:02:37] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:56424]
frigate  | [2022-09-07 02:02:47] watchdog.driveway              INFO    : No frames received from driveway in 20 seconds. Exiting ffmpeg...
frigate  | [2022-09-07 02:02:47] watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...
frigate  | [2022-09-07 02:02:50] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51066]
frigate  | [2022-09-07 02:02:55] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:45176]
frigate  | [2022-09-07 02:03:17] watchdog.driveway              INFO    : FFmpeg didnt exit. Force killing...
frigate  | [2022-09-07 02:03:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:03:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:03:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:03:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:03:17] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
frigate  | [2022-09-07 02:03:27] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.
frigate  | [2022-09-07 02:03:27] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-09-07 02:03:47] watchdog.driveway              INFO    : No frames received from driveway in 20 seconds. Exiting ffmpeg...
frigate  | [2022-09-07 02:03:47] watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...
frigate  | [2022-09-07 02:03:55] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:45176]
frigate  | [2022-09-07 02:04:00] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51724]
frigate  | [2022-09-07 02:04:17] watchdog.driveway              INFO    : FFmpeg didnt exit. Force killing...
frigate  | [2022-09-07 02:04:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:04:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:04:17] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
frigate  | [2022-09-07 02:04:27] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.
frigate  | [2022-09-07 02:04:27] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-09-07 02:04:47] watchdog.driveway              INFO    : No frames received from driveway in 20 seconds. Exiting ffmpeg...
frigate  | [2022-09-07 02:04:47] watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...
frigate  | [2022-09-07 02:05:00] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51724]
frigate  | [2022-09-07 02:05:05] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:58226]
frigate  | [2022-09-07 02:05:17] watchdog.driveway              INFO    : FFmpeg didnt exit. Force killing...
frigate  | [2022-09-07 02:05:17] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
frigate  | [2022-09-07 02:05:17] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
frigate  | [2022-09-07 02:05:27] watchdog.driveway              ERROR   : Ffmpeg process crashed unexpectedly for driveway.
frigate  | [2022-09-07 02:05:27] watchdog.driveway              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | [2022-09-07 02:05:47] watchdog.driveway              INFO    : No frames received from driveway in 20 seconds. Exiting ffmpeg...
frigate  | [2022-09-07 02:05:47] watchdog.driveway              INFO    : Waiting for ffmpeg to exit gracefully...

FFprobe output from your camera

root@edcaa9ad0fcf:/opt/frigate# ffprobe "rtsp://xxxxx:xxxxx@driveway.cam.home:554/cam/realmonitor?channel=1&subtype=0"
ffprobe version n5.1-2-g915ef932a3-20220731 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=aarch64-ffbuild-linux-gnu- --arch=aarch64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --disable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, rtsp, from 'rtsp://xxxxx:xxxxx@driveway.cam.home:554/cam/realmonitor?channel=1&subtype=0':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt470bg/bt470bg/bt709, progressive), 2688x1520, 10 fps, 10 tbr, 90k tbn
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s

Frigate stats

{
  "detection_fps": 0,
  "detectors": {
    "coral": {
      "detection_start": 0,
      "inference_speed": 10,
      "pid": 215
    }
  },
  "driveway": {
    "camera_fps": 0,
    "capture_pid": 227,
    "detection_fps": 0,
    "pid": 221,
    "process_fps": 0,
    "skipped_fps": 0
  },
  "service": {
    "latest_version": "0.10.1",
    "storage": {
      "/dev/shm": {
        "free": 133.9,
        "mount_type": "tmpfs",
        "total": 134.2,
        "used": 0.3
      },
      "/media/frigate/clips": {
        "free": 9131015.5,
        "mount_type": "cifs",
        "total": 19859668.5,
        "used": 10728652.9
      },
      "/media/frigate/recordings": {
        "free": 9131015.5,
        "mount_type": "cifs",
        "total": 19859668.5,
        "used": 10728652.9
      },
      "/tmp/cache": {
        "free": 954,
        "mount_type": "tmpfs",
        "total": 1000,
        "used": 46
      }
    },
    "temperatures": {},
    "uptime": 1041,
    "version": "0.11.0-c461c9e"
  }
}

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

Dahua IPC-T5442T-ZE

Any other information that may be helpful

No response

NickM-27 commented 2 years ago

It seems you're running on the RPi and using hwaccel. Have you tried it without hwaccel?

kylehendricks commented 2 years ago

That was it!

I'm still thoroughly confused as to why this suddenly started happening. I even tried beta7/rc1 and they are no longer working either. My other 1080p Dahua also still works with hwaccel on...

Is this a known thing that hwaccel sometimes just doesn't work on a rpi4?

NickM-27 commented 2 years ago

What is your GPU memory set to? Odds are you have another program using too much memory so the Pis GPU is starving and unable to process the frames.

kylehendricks commented 2 years ago

My gpu_mem is set to 256 according to my /boot/config.txt

This pi4 is solely being used for frigate.

As I was typing this, one thing I remember did change recently is i ran a apt full-upgrade on the pi recently. That has to be related to this sudden issue.

NickM-27 commented 2 years ago

Maybe try increasing it to 512mb

kylehendricks commented 2 years ago

I was thinking the max was 256 for some reason. Tried bumping to 512 and rebooting, exact same behavior when hwaccel is on.

I just looked at my /var/log/apt/history.log and saw my most recent apt upgrade was right around when the issue started happening. Here's what was updated:

raspberrypi-bootloader:arm64 (1:1.20220811-1, 1:1.20220830-1)
libcamera-apps-lite:arm64 (0~git20220707+35266e8-1, 0~git20220830+1bf0cca-1)
libcamera0:arm64 (0~git20220705+f30ad033-1, 0~git20220826+3fad116f-1)
raspberrypi-kernel:arm64 (1:1.20220811-1, 1:1.20220830-1)
linux-libc-dev:arm64 (1:1.20220811-1, 1:1.20220830-1)

Could be the kernel or bootloader update?

kylehendricks commented 2 years ago

I don't know much about ffmpeg and hardware acceleration but I searched the raspberry pi kernel issues for the kernel version I just updated to (5.15.61-v8+) and found a couple recent issues that may be related: https://github.com/raspberrypi/linux/issues/5150 https://github.com/raspberrypi/linux/issues/5166

kylehendricks commented 2 years ago

ffprobe output for a camera that's currently working with hwaccel:

ffprobe version n5.1-2-g915ef932a3-20220731 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=aarch64-ffbuild-linux-gnu- --arch=aarch64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --disable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, rtsp, from 'rtsp://xxxxx:xxxxxx@south-garage.cam.home:554/cam/realmonitor?channel=1&subtype=0':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.200000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 10 fps, 10 tbr, 90k tbn
NickM-27 commented 2 years ago

We've seen other users complain that the latest RPi kernel broke hwaccel as well

LaurenceGough commented 2 years ago

Yeah same issue here sadly. :( Turning off hw Accel works for now but the poor thing is deffo working harder even with just two cameras setup.

It happens with rc1 and rc2 for me. I'm using two different hikvision cameras sadly neither work. H264.

If I find a better fix I'll be sure to share

idontcare99999 commented 2 years ago

Same here. Fresh install of Raspberry Pi OS Lite 2022-09-06 with kernel 5.15.61-v8+ on a Raspberry Pi 4 Model B Rev 1.2 running docker and only green screen when using hwaccel_args:

Just tried crzynik/frigate:ffmpeg-update and same result.

Seems the issue has been identified: https://github.com/raspberrypi/linux/issues/5150

idontcare99999 commented 2 years ago

and the fix is in... https://github.com/raspberrypi/linux/issues/5150#issuecomment-1247922065

joe248 commented 2 years ago

I'm experiencing this issue as well since updating the PI kernel and trying to figure out a solution. My understanding is that:

  1. The kernel update broke things, but the issue is with FFmpeg
  2. The fix is in for the 4.3.4 version of FFmpeg packaged with the PI, but hasn't been released yet, though this won't solve the issue because Frigate uses a different version of FFmpeg inside the Docker container
  3. Frigate is using a BtbN build of FFmpeg 5.1 which appears to be auto-built from the official FFmpeg repo

After looking at the fix that @idontcare99999 mentioned, I checked the official FFmpeg repo and it doesn't appear to contain the fix. Has anyone else managed to get hwaccel working again other than by downgrading to the previous kernel (which I'm not opposed to but have never done before)?

LaurenceGough commented 2 years ago

I'm experiencing this issue as well since updating the PI kernel and trying to figure out a solution. My understanding is that:

  1. The kernel update broke things, but the issue is with FFmpeg
  2. The fix is in for the 4.3.4 version of FFmpeg packaged with the PI, but hasn't been released yet, though this won't solve the issue because Frigate uses a different version of FFmpeg inside the Docker container
  3. Frigate is using a BtbN build of FFmpeg 5.1 which appears to be auto-built from the official FFmpeg repo

After looking at the fix that @idontcare99999 mentioned, I checked the official FFmpeg repo and it doesn't appear to contain the fix. Has anyone else managed to get hwaccel working again other than by downgrading to the previous kernel (which I'm not opposed to but have never done before)?

Make sure you do a disk clone or otherwise full backup before messing with the kernel to save yourself many hours of work...

Yeah, I'm a complete Linux newbie so I most likely did something very wrong, It'd be nice to get HWaccel working again but I do like having Home assistant and Frigate running ;).

If anyone finds a way to downgrade the kernel please let me know. Thank you. Never again will I update without a full, tested backup.

joe248 commented 2 years ago

I used the following process to downgrade the kernel from 5.15.61-v8+ to 5.15.32-v8+ and hwaccel is working again:

  1. Check /var/log/apt to find last kernel date version before upgrade. For me it was 1.20220331-1.
  2. Check https://github.com/raspberrypi/rpi-firmware/commits/master to find commit hash for that date (a54fe46c85fd4a2155f2282454bee3c2a3d5b5eb)
  3. Power down the PI, put the SD card into a computer, and create a backup image. For Mac OS I used: sudo dd bs=4m if=/dev/disk2 of=raspbian.img
  4. Put the SD card back into the PI and boot. Run sudo rpi-update a54fe46c85fd4a2155f2282454bee3c2a3d5b5eb
  5. Reboot
jherby2k commented 2 years ago

Seems to affect Home Assistant OS (HASSOS) also (on Pi 4). Or at least this seems to be what i'm experiencing - new install, hwaccel has never worked.

LaurenceGough commented 2 years ago

I am not sure if I should downgrade the kernel using Joe's instructions as kindly posted above or try to build the Pi specific FFMPEG build here in a test branch:

raspberrypi/linux#5150 (comment)

I am not sure how long we would have to wait for the next Pi ffmpeg update. Has anyone here tried to build this specific branch version? As of today hwaccel for Frigate on the Pi 4 fully updated still doesn't work unfortunately.

For now I am making do without hwaccel but the Pi is certainly struggling even with just two cameras I'd imagine this would be a nightmare for people with more.

I am not seeing too many people reporting this issue, I take it many are not updating their Pi's? :o

I might give the sudo rpi-update a54fe46c85fd4a2155f2282454bee3c2a3d5b5eb a go after making a full backup.

idontcare99999 commented 2 years ago

I'm hoping I can wait it out...

I have built the fixed ffmpeg inside a few different docker containers but could never get it to work within the frigate container.

jherby2k commented 2 years ago

I'm surprised more people don't report this issue as well - anyone running HA on a Raspberry Pi via HA OS (So the most popular install type) would be experiencing this AFAICT.

LaurenceGough commented 2 years ago

The same as well for Raspberry Pi OS and also DietPi OS too.

sparkydave1981 commented 2 years ago

I have the same issue however I'm running HAOS on an i7 NUC rather than a RPi. What info can I share to help with a fix?

many errors like this:

[2022-10-19 09:24:46] frigate.video                  ERROR   : front_door: Unable to read frames from ffmpeg process.
[2022-10-19 09:24:46] frigate.video                  ERROR   : front_door: ffmpeg process is not running. exiting capture thread...
[2022-10-19 09:24:46] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
[2022-10-19 09:24:46] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture
NickM-27 commented 2 years ago

@sparkydave1981 make your own issue, that is unrelated to this one if it is not an RPi

domhaas commented 2 years ago

The same as well for Raspberry Pi OS and also DietPi OS too.

Yeah - can confirm this. Tried it today on DietPi.

Salvora commented 2 years ago

Same issue here: Raspberry Pi4 with Raspberry Pİ OS Linux rpi 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux frigate 0.11.1-2EADA21

domhaas commented 2 years ago

The same as well for Raspberry Pi OS and also DietPi OS too.

Yeah - can confirm this. Tried it today on DietPi.

Update: I've simply tried it with activated privileged-mode and it works now. Maybe I've missed to attach a device, but tried really everything I've found so far.

I don't know if those boot-settings are really required, but maybe it helps somebody:

/etc/boot.txt

(...)
arm_64bit=1
dtoverlay=disable-wifi
dtoverlay=rpivid-v4l2
dtoverlay=vc4-fkms-v3d

config.yml

    ffmpeg:
      hwaccel_args:
        - -c:v
        - h264_v4l2m2m
idontcare99999 commented 2 years ago

FWIW... I think I have this working now by statically compiling the dev/5.1.2/rpi_import_1 branch in a new Ubuntu container and then copying it over to my frigate container, so that should be encouraging.

What's interesting is I have no green screen for the first 3 cameras, but then once I activate 4 one of them will begin throwing errors like this:

[flv @ 0x45a6800] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found Error while decoding stream #0:0: Resource temporarily unavailable Last message repeated 79 times [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found [h264_v4l2m2m @ 0x459cf30] V4L2 capture poll unexpected timeout: events=0x106 [h264_v4l2m2m @ 0x459cf30] Poll thinks src Q has space; none found Finishing stream 1:0 without any data written to it. [swscaler @ 0x450f2e0] [swscaler @ 0x4525760] deprecated pixel format used, make sure you did set range correctly [swscaler @ 0x450f2e0] [swscaler @ 0x45326f0] deprecated pixel format used, make sure you did set range correctly [swscaler @ 0x450f2e0] [swscaler @ 0x45456b0] deprecated pixel format used, make sure you did set range correctly [swscaler @ 0x450f2e0] [swscaler @ 0x4552640] deprecated pixel format used, make sure you did set range correctly [swscaler @ 0x450f2e0] [swscaler @ 0x45bf6f0] deprecated pixel format used, make sure you did set range correctly [flv @ 0x45a6800] Failed to update header with correct duration. [flv @ 0x45a6800] Failed to update header with correct filesize.

NickM-27 commented 2 years ago

What's interesting is I have no green screen for the first 3 cameras, but then once I activate 4 one of them will begin throwing errors like this:

You probably need to increase the Pis GPU memory

idontcare99999 commented 2 years ago

What's interesting is I have no green screen for the first 3 cameras, but then once I activate 4 one of them will begin throwing errors like this:

You probably need to increase the Pis GPU memory

Good call! I lowered it once hwaccel stopped working and I reduced to one cam. Now all 4 are running with -c:v h264_v4l2m2m.

jherby2k commented 2 years ago

That's awesome! I've very eager to test this... can you make your binary available and/or provide instructions?

idontcare99999 commented 2 years ago

I plan to test more tonight to confirm all is good then will share whatever I remember of my instructions. I'm learning as I go so things got pretty messy, but I do at least have my config file.

I haven't noticed a dramatic reduction in CPU, but then again I'm using the second stream at D1 resolution.

LaurenceGough commented 2 years ago 
(...)
arm_64bit=1
dtoverlay=disable-wifi
dtoverlay=rpivid-v4l2
dtoverlay=vc4-fkms-v3d

config.yml

    ffmpeg:
      hwaccel_args:
        - -c:v
        - h264_v4l2m2m

I have the same (minus dtoverlay=vc4-fkms-v3d which I believe is for 3d applications only?).

The latest version of DietPi which essentially is Raspberry OS still is just green :(.

I am running in privileged mode in Portainer.

I wonder how yours is working.

Are you using DietPi, Portainer and/or Docker?

Did you run rpi-update?

"rpi-update is a command line application that will update your Raspberry Pi OS kernel and VideoCore firmware to the latest pre-release versions.

WARNING | Pre-release versions of software are not guaranteed to work. You should not use rpi-update on any system unless recommended to do so by a Raspberry Pi engineer. It may leave your system unreliable or even completely broken. It should not be used as part of any regular update process."

It's tempting....

NickM-27 commented 2 years ago

I am running in privileged mode in Portainer.

@LaurenceGough is portainer itself also run in privileged mode?

LaurenceGough commented 2 years ago

I am running in privileged mode in Portainer.

@LaurenceGough is portainer itself also run in privileged mode?

I believe so, it's running as the root user (I know, I know, not ideal but it's pretty protected otherwise for now). The Frigate container is 100% set to privileged also which I believe is the only thing I do can do for the Frigate container. It worked before :(.

roger- commented 2 years ago

I plan to test more tonight to confirm all is good then will share whatever I remember of my instructions. I'm learning as I go so things got pretty messy, but I do at least have my config file.

I haven't noticed a dramatic reduction in CPU, but then again I'm using the second stream at D1 resolution.

I'd appreciate a binary too.

idontcare99999 commented 2 years ago

System was not happy after a few hours. Couldn't even restart the container - actually needed to restart the server. Tonight I'm trying it with hevc to see if it stays up overnight. I can tell already the CPU usage is much lower, so maybe I wasn't doing something correctly with h264.

roger- commented 2 years ago

@idontcare99999 Maybe you should report the instability in this thread?

idontcare99999 commented 2 years ago

I'm pretty happy with what I've done and how it's been running, so here's my story.

I have 4 cameras outputting HEVC at D1 resolution without hiccups for the past 24 hours. Everything looks good to me, except you can't use Firefox since it won't read HEVC via rtmp.

To get here, I needed to patch flv to accept hevc and then statically compile ffmpeg.

I created a new container just for this purpose using the latest ubuntu.

I pulled sources for ffmpeg branch dev/5.1.2/rpi_import_1, fdk-aac, x264 (since that was my original plan), zimg, drm, and systemd.

I followed various guides and improvised where I needed to including: https://github.com/jc-kynesim/rpi-ffmpeg/blob/dev/5.1.2/rpi_import_1/pi-util/BUILD.txt https://pimylifeup.com/compiling-ffmpeg-raspberry-pi/ and most importantly https://segmentfault.com/a/1190000040129616/en

I learned I needed to statically compile libdrm and libudev (using systemd) so I could build a static ffmpeg to copy into my frigate container.

If you also want h265 like I'm using now, then this site will show you the way: https://github.com/kn007/patch/blob/master/ffmpeg-let-rtmp-flv-support-hevc-h265-opus.patch

I'm using dtoverlay=vc4-kms-v3d-pi4 and also dropped gpu_mem to 16 since hevc uses cma and all seems good running 5.15.74-v8+ on bullseye with hwaccel_args: -hwaccel drm in my frigate.yml.

Finally, here's my config which is ugly but since it ain't broke I ain't fixin' it...

Good luck!

./configure \ --extra-cflags="-I/usr/local/include -I/usr/include -I/usr/include/drm -I/usr/include/libdrm -static" \ --extra-ldflags="-L/usr/local/lib -L/usr/lib/aarch64-linux-gnu/ -L/lib/aarch64-linux-gnu/ -static" \ --pkg-config-flags="--static" \ --extra-libs="-lpthread -lm -lz -latomic" \ --extra-ldexeflags="-static" \ --enable-neon \ --enable-hwaccels \ --enable-version3 \ --enable-gpl \ --enable-nonfree \ --disable-shared \ --enable-static \ --disable-debug \ --disable-doc \ --disable-ffplay \ --disable-indev=sndio \ --disable-outdev=sndio \ --arch=arm64 \ --enable-libx264 \ --target-os=linux \ --enable-libzimg \ --enable-libfdk-aac \ --enable-pthreads \ --disable-mmal \ --disable-stripping \ --disable-thumb \ --enable-sand \ --enable-libv4l2 \ --enable-v4l2-request \ --enable-libdrm \ --enable-libudev \ --enable-vout-drm \ --enable-hardcoded-tables \ --extra-version="rpi"

roger- commented 2 years ago

@idontcare99999 thanks for providing those details, should be helpful to the next person. Any chance you could provide your final binary?

jherby2k commented 2 years ago

yes, that's very helpful but looks like a whole lot of work - would love if you'd just upload the binary somewhere for us to try.

roger- commented 2 years ago

I was able to compile the 5.1.2/rpi_import_1 branch according to the build instructions.

Unfortunately it didn't fix the issue for me, but posting the (arm64) binary in case someone else wants to give it a shot: ffmpeg.zip

Salvora commented 2 years ago

I tried to compile a ffmpeg that can fix this but couldn't manage either. @idontcare99999 could you share your binary file?

giga89 commented 2 years ago

I was able to compile the 5.1.2/rpi_import_1 branch according to the build instructions.

Unfortunately it didn't fix the issue for me, but posting the (arm64) binary in case someone else wants to give it a shot: ffmpeg.zip

Is it ok for hassio? Where I have to put this file?

roger- commented 2 years ago

I was able to compile the 5.1.2/rpi_import_1 branch according to the build instructions. Unfortunately it didn't fix the issue for me, but posting the (arm64) binary in case someone else wants to give it a shot: ffmpeg.zip

Is it ok for hassio? Where I have to put this file?

I used Docker to substitute my binary for the included one (instructions are in the documentation).

jherby2k commented 2 years ago

I tried @roger- 's binary but it has issues. These are the steps I performed:

  1. Copied ffmpeg from the zip file to my config share
  2. SSH into HASSOS
  3. Ran docker exec -it addon_ccab4aaf_frigate-fa bash to enter the frigate container
  4. Ran rm /usr/lib/btbn-ffmpeg/bin/ffmpeg so the next command would work
  5. Ran cp /config/ffmpeg /usr/lib/btbn-ffmpeg/bin/ffmpeg to replace the included one
  6. Ran chmod 755 /usr/lib/btbn-ffmpeg/bin/ffmpeg and chown 1001:121 /usr/lib/btbn-ffmpeg/bin/ffmpeg
  7. Did NOT restart addon. This resets the container and wipes your changes. I'm not sure yet how to persist them.

At this point the accelerated feed throws an error:

ffmpeg: error while loading shared libraries: libxcb-shm.so.0: cannot open shared object file: No such file or directory

Running ldd ffmpeg shows a lot of additional dependencies not present in the (much larger) default ffmpeg. Your binary doesn't appear to be statically linked, unfortunately.

default ffmpeg:

    linux-vdso.so.1 (0x0000007f8bd36000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f866b3000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f8669f000)
    librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f86687000)
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f86656000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f864e1000)
    /lib/ld-linux-aarch64.so.1 (0x0000007f8bd06000)

@roger- 's ffmpeg:

    linux-vdso.so.1 (0x0000007fbddc1000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007fbc406000)
    libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007fbc3ce000)
    libxcb-shm.so.0 => not found
    libxcb-shape.so.0 => not found
    libxcb-xfixes.so.0 => /usr/lib/aarch64-linux-gnu/libxcb-xfixes.so.0 (0x0000007fbc3b6000)
    libasound.so.2 => not found
    libSDL2-2.0.so.0 => not found
    libsndio.so.7.0 => not found
    libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007fbc393000)
    libXv.so.1 => not found
    libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007fbc246000)
    libXext.so.6 => not found
    libva.so.2 => /usr/lib/aarch64-linux-gnu/libva.so.2 (0x0000007fbc20b000)
    libbz2.so.1.0 => /lib/aarch64-linux-gnu/libbz2.so.1.0 (0x0000007fbc1ea000)
    libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007fbc1c0000)
    liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007fbc18a000)
    libudev.so.1 => /usr/lib/aarch64-linux-gnu/libudev.so.1 (0x0000007fbc153000)
    libva-drm.so.2 => /usr/lib/aarch64-linux-gnu/libva-drm.so.2 (0x0000007fbc140000)
    libva-x11.so.2 => not found
    libvdpau.so.1 => not found
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007fbc10f000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007fbbf9a000)
    /lib/ld-linux-aarch64.so.1 (0x0000007fbdd91000)
    libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007fbbf86000)
    libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007fbbf70000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007fbbf5c000)
    libbsd.so.0 => /usr/lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007fbbf37000)
    libmd.so.0 => /usr/lib/aarch64-linux-gnu/libmd.so.0 (0x0000007fbbf1b000)
roger- commented 2 years ago

@jherby2k I actually expected the binary to be static (according to the build instructions) but I didn't check and it appears you're right.

I think the configure script needs both the --enable-static and --disable-shared options; will try compiling and post result here if it works.

idontcare99999 commented 2 years ago

Nice to see all the activity!

Need to compile ffmpeg with --pkg-config-flags="--static" to build a static binary.

I'm still working on mine, but can tell you h264 is not looking good while h265 is very impressive although has compatibility challenges.

I'll share once I have something stable.

roger- commented 2 years ago

Nice to see all the activity!

Need to compile ffmpeg with --pkg-config-flags="--static" to build a static binary.

I'm still working on mine, but can tell you h264 is not looking good while h265 is very impressive although has compatibility challenges.

I'll share once I have something stable.

Tried that parameter too but no luck, thinks it's an issue with my system.

jherby2k commented 2 years ago

@NickM-27 there are a lot of people in here who would be happy to test a fixed binary, but we seem to be missing the expertise required to build it. Any help appreciated! Would love to see this fixed in the next release

idontcare99999 commented 2 years ago

Here's my latest compile.

It works for both 264 and 265 but h264_v4l2m2m fails for me after running for a bit.

https://www.dropbox.com/s/chxy90fpgif6rlq/ffmpeg.1104.gz?dl=0

Have fun!

jherby2k commented 2 years ago

Thanks, i'm running it now! So far stable at h264 (my cameras don't do h265). However I'm not sure there is any advantage to the "acceleration". Either its not actually running an optimized path, or its just completely underwhelming. I'm at 50% CPU running 3 cameras at 960x576 resolution. Looks like HD isn't in the cards for me.

I'd love to not have to run through 5 steps every time i restart... anyone know how to persist changes to the container?