[Camera Support]: What should I set the detect width & height to?

[WizBangCrash]

Describe the problem you are having

Firstly, thanks for such a great application. After spending two weeks evaluating both Shinobi and ZoneMinder and getting more and more frustrated with them. I came across frigate and haven't looked back :-)

I've a question about the values to use the in the detect element of the camera configuration. Should these be set to the actual values of the camera video feed i.e. 3840x2160 in my case? Or should they be set to the width and height that you want the detect engine to use e.g. 1280x720

It's not clear in the documentation and I may be overthinking it, but when I read the page on detect resolution I wondered if I am supposed to set lower numbers here to help fit the 320x320 model.

Any light someone can shed here would be really useful.

Version

0.11.0 Beta 4

Frigate config file

cameras:
  street:
    ffmpeg:
      inputs:
        - path: rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@reolink-front.lan:554/h265Preview_01_main
          roles:
            - record
            - detect
        - path: rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@reolink-front.lan:554/h264Preview_01_sub
          roles:
            - rtmp
    detect:
      enabled: true
      width: 3840
      height: 2160
      fps: 5
    record:
      enabled: true

Relevant log output

N/A

FFprobe output from your camera

ffprobe version 4.3.2-Jellyfin Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-gpl --enable-version3 --enable-static --enable-libfontconfig --enable-fontconfig --enable-gmp --enable-gnutls --enable-libass --enable-libbluray --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --arch=amd64 --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-vdpau --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvenc --enable-nvdec --enable-ffnvcodec
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, rtsp, from 'rtsp://<user>:<password>@reolink-front.lan:554/h265Preview_01_main':
  Metadata:
    title           : Session streamed by "preview"
    comment         : h265Preview_01_main
  Duration: N/A, start: 0.000188, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 24.92 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp

Frigate stats

No response

Operating system

Debian

Install method

Docker Compose

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Reolink RLC-810A and RLC-811A

Any other information that may be helpful

Also, for reference to anyone else out there, I have three Reolink cameras (2 x RLC-810A + RLC-811A) and I have had no issues using the default settings for ffmpeg input_args. I have the latest firmware on all of them (v3.1.0), I think this section does not apply to the newer models.

[NickM-27]

So the main point of concern with the detect resolution is CPU utilization

if detect -> width and detect -> height are different than the camera

in this case, frigate will resize the camera feed that is used for detection to the width and height specified which of course requires CPU usage (continuously) and can increase overall CPU use.

if detect -> width and detect -> height match that of your camera

in this case, frigate will not need to use CPU to resize the camera feed, however feeds that are high resolution (especially as high as yours) will add a considerable amount of additional CPU load when running motion detection (as there are more pixels to compare).

So, what to do?

It's all about balance. The ideal solution is to have a camera with multiple streams that have a configurable resolution, so you can have a full resolution stream for recording and a smaller feed for detection which you can match frigate config to.

If that option is not available then the main thing to consider is the size of objects you are expecting to detect. As the docs discuss, the default models have a region size of 320 x 320 so if you look at that size compared to the resolution of the camera feed, that is pretty close to the smallest object you want to detect. If you are looking to detect cats or other smaller objects then a higher resolution is optimal. Then, also considering how much CPU headroom you have, can choose a resolution.

Personally, since I am running a very capable computer and my cameras substream is too low resolution (and not configurable) I have 2560x1440 for the record and then I take that stream and cut it in half to 1080x720 which has worked well for me. Frigate is usually around 5-8% CPU (with a coral) so I have no issues with that.

[WizBangCrash]

@NickM-27 Thanks for the detailed explanation. That makes it a lot clearer. I have the same issue with the Reolink cameras. The sub stream is too low quality so I have to use the main hires stream. I will do some experimentation and watch the CPU overhead. :-) I'm using an old iMac, so I have plenty of CPU to play with. I'm stuck with using the CPU as the USB Corals are out of stock everywhere at the moment and eBay prices are $200+

[themagic314]

@WizBangCrash Saw on reddit that rs-online UK have them in stock for around £50.

[WizBangCrash]

@obj Thanks for that. It definitely looks like supply is easing for that product. UK Farnell have them in stock now too. I shall be placing my order soon :-)

[solarsparq]

So the main point of concern with the detect resolution is CPU utilization

if detect -> width and detect -> height are different than the camera

in this case, frigate will resize the camera feed that is used for detection to the width and height specified which of course requires CPU usage (continuously) and can increase overall CPU use.

if detect -> width and detect -> height match that of your camera

in this case, frigate will not need to use CPU to resize the camera feed, however feeds that are high resolution (especially as high as yours) will add a considerable amount of additional CPU load when running motion detection (as there are more pixels to compare).

So, what to do?

It's all about balance. The ideal solution is to have a camera with multiple streams that have a configurable resolution, so you can have a full resolution stream for recording and a smaller feed for detection which you can match frigate config to.

If that option is not available then the main thing to consider is the size of objects you are expecting to detect. As the docs discuss, the default models have a region size of 320 x 320 so if you look at that size compared to the resolution of the camera feed, that is pretty close to the smallest object you want to detect. If you are looking to detect cats or other smaller objects then a higher resolution is optimal. Then, also considering how much CPU headroom you have, can choose a resolution.

Personally, since I am running a very capable computer and my cameras substream is too low resolution (and not configurable) I have 2560x1440 for the record and then I take that stream and cut it in half to 1080x720 which has worked well for me. Frigate is usually around 5-8% CPU (with a coral) so I have no issues with that.

I just wanted to say thank you for explaining this. I am offered extremely low-resolution sub-stream options, so I wanted to use my main stream.. but as you did, cut its detect resolution in half. Hopefully this will help with detection at longer ranges & keep CPU tamed. I will be purchasing a Coral to supplement my setup running on Docker on an ODROID-H2. I am a Shinobi migrant. Thank you again!

[simondsmason]

My Reolink substream is 640x360. It does detect cats every so often but I definitely don't think it is happening most of the time. From reading this thread it seems that the small size of the stream is probably an issue. So best to take my main stream (2304x1296) and cut that in half to 1152x648?

[NickM-27]

If you have hwaccel then that's not a problem to do, I'd suggest a standard 1280x720 for 16:9