search

geerlingguy / pi-nvr

Raspberry Pi NVR for home CCTV recording.
209 stars 22 forks source link

Optimize Frigate Performance / Utility #8

Open geerlingguy opened 6 months ago

geerlingguy commented 6 months ago

Frigate is looking like the head-and-shoulders winner for running on a Pi (at least Pi 5), especially in tandem with Coral. Would love to see other 'neural' TPUs supported too (Hailo... maybe the one built into some Rockchip SoCs?).

But I'm seeing that it won't scale too drastically with H.265, H.264, 4K, multiple cameras... it starts eating CPU just transcoding stuff and translating it for internal processing (ffmpeg is eating up 150% CPU constantly while just processing one 4K stream at H.265).

I want to make sure the Pi 5's built-in hardware decoding support is used where possible, and also make sure the TPU and any other little performance optimization is used to its fullest extent. And also streamline the deployment a bit, especially since I'd like one configuration to run both my home setup and my office (with different types and numbers of cameras).

Also would like to make it so I could easily swap over to a standard Linux PC at some point or maybe even try running it with Docker on my Mac (is that doable?), all with the same Ansible playbook.

For some inspiration, see Raspberry Pi 3/4 with ffmpeg.hwaccel_args: preset-rpi-64-h264

Things to investigate:

Follow-up to #4.

geerlingguy commented 5 months ago

Using the substream (640x480, H265 — the camera also allows for MJPEG if desired) for detect, I'm able to view the lower-res in browser, while clips and recordings are still done at full 4K resolution:

cameras:
  basement:
    ffmpeg:
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -tag:v hvc1 -bsf:v hevc_mp4toannexb -c:a aac
        rtmp: -c:v copy -c:a aac -f flv
      inputs:
        - path: "rtsp://jgeerling:{FRIGATE_RTSP_PASSWORD}@cam02.geerli.net:554/H264/ch1/sub/av_stream"
          roles:
            - detect
        - path: "rtsp://jgeerling:{FRIGATE_RTSP_PASSWORD}@cam02.geerli.net:554/H264/ch1/main/av_stream"
          roles:
            - record

The CPU usage is down to like 40% overall (much less than one full core), and FPS for ffmpeg/Capture are capped at 5. The recordings and clips are all at 15 fps.

So... don't use the 4K stream for detect, only for record :)

geerlingguy commented 5 months ago

I also brought home a Pineberry Pi HatDrive! Bottom and am 3D Printing this case, so I'm able to run everything off a 1TB NVMe drive over PCIe. Nice and fast.

geerlingguy commented 5 months ago

Power consumption for my Pi 5 with an attached Coral USB accelerator and NVMe via Pineberry HatDrive! Bottom (booting off NVMe, no microSD card inserted), with two Annke 4K cameras, running event detection and 24x7 recording on both, averages 5.2W (over the course of 24 hours).

Want to also test with CM4 and with the Mini PC. Maybe also some 1080p recording testing (should be nearly identical on each system vs 4K though, since I'm processing the 640x360 substream, and recording the HD/4K stream directly to disk).

platypus11 commented 3 months ago

Hi Jeff, Very many thanks for this. I've recently switched to frigate from surveillance station as the person detection is super reliable with frigate and not susceptible to false alerts caused by wind, rain etc. I'm running 5 cameras on a rpi5 with a coral usb stick and had no cpu% issues at all. I'm just curious- have you made any progress with the hardware acceleration for H.265 decode? I currently don't have it set up. I've had a poke around on the web and haven't been able to find anything definitive

geerlingguy commented 3 months ago

What I wound up doing was using a substream for preview, which means the web UI can stream the video (and Pi can do motion detect) using very little CPU without worrying about decode. Then for playback and export, it just handles the H.265 files straight to the browser.

The slowest part is exporting footage—ffmpeg eats up a lot of CPU doing that with my 4K streams.

platypus11 commented 3 months ago

Nice!

On Tue, 5 Mar 2024 at 01:06, Jeff Geerling @.***> wrote:

What I wound up doing was using a substream for preview, which means the web UI can stream the video (and Pi can do motion detect) using very little CPU without worrying about decode. Then for playback and export, it just handles the H.265 files straight to the browser.

The slowest part is exporting footage—ffmpeg eats up a lot of CPU doing that with my 4K streams.

— Reply to this email directly, view it on GitHub https://github.com/geerlingguy/pi-nvr/issues/8#issuecomment-1977758356, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYZFA4PYOBYZ7OXGG62YDS3YWULBXAVCNFSM6AAAAABBKZ5EOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXG42TQMZVGY . You are receiving this because you commented.Message ID: @.***>

w-marco commented 2 months ago

I just stumbled across the corresponding video and found this repo. Went onto the journey to setup frigate myself a few weeks ago. Have you tried with go2rtc and restream to compare how CPU usage does?

It also seems to be the preffered method by the frigate developers rather than directly using the RTSP streams inside ffmpeg.

A sample would look like this (the RTSP URLs are Reolink but can be adapted to match other camera makers):


go2rtc:
  streams:
    basement:
      - rtsp://USER:PASS@CAM_IP:554/h264Preview_01_main
      - "ffmpeg:basement#audio=opus"
    basement_sub:
      - rtsp://USER:PASS@CAM_IP:554/h264Preview_01_sub
  webrtc:
    candidates:
      - FRIGATE_IP:8555
      - stun:8555

cameras:
  basement:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/basement
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/basement_sub
          roles:
            - detect

One could then also use the ffmpeg input_args preset named 'preset-rtsp-restream'.

danluckner commented 6 days ago

@w-marco

I have a Reolink Doorbell PoE and have been trying to get my RPi5 working well with your configuration idea, but I'm getting ffmpeg errors. Tried both h264 and h265 streams from the Doorbell, but cannot get it to work:

2024-06-24 16:44:16.669821843 [2024-06-24 16:44:14] frigate.video ERROR : reolink: ffmpeg process is not running. exiting capture thread... 2024-06-24 16:44:44.182194872 [2024-06-24 16:44:44] ffmpeg.reolink.record ERROR : [tcp @ 0xaaaad9da9e60] Connection to tcp://FRIGATE_IP:8554?timeout=5000000 failed: Connection refused 2024-06-24 16:44:44.182200075 [2024-06-24 16:44:44] ffmpeg.reolink.record ERROR : rtsp://FRIGATE_IP/reolink: Connection refused

Note: I tried this in a Proxmox homelab machine with x64 processor and it works fine, so I'm sure my configuration is correct. I assume it is the inability of the RPi5 dealing with H265 decoding.

geerlingguy commented 5 days ago

@danluckner - This issue isn't probably the best location for debugging Reolink issues, but the Pi 5 should be able to handle H.265 okay. It's more likely a configuration issue; I would consider posting your full config maybe to the frigate issue queue or just double checking every value, especially to make sure it's all spelled correctly.

See for example https://github.com/blakeblackshear/frigate/issues/699 or https://github.com/blakeblackshear/frigate/issues/8505

danluckner commented 5 days ago

@geerlingguy thanks! I will check the references and post it in the right forum, if needed. I will come back if I have a solution.

danluckner commented 4 days ago

Hey guys, I think I was able to fix it. I let it run overnight to make sure it's stable and it seems so. The main changes were: