search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.27k stars 1.66k forks source link

[Config Support]: Detect resolution defaults are wrong in 0.13.0-beta4 #8488

Closed haimgel closed 10 months ago

haimgel commented 10 months ago

Describe the problem you are having

The "detect" resolution seems to default to the original camera stream resolution, and not to 1280x720 as written in the docs. I saw that the zone coordinates were all wrong in the "debug" screen, and the zones events seemed not to trigger as expected. As soon as I added 

detect:
  width: 1280
  height: 720

the zone boundaries were shown correctly again, and the detection seems to work fine as well.

Version

0.13.0-65e3e67

Frigate config file

mqtt:
  host: xxx

birdseye:
  enabled: False

database:
  path: /media/frigate/frigate.db

audio:
  enabled: True
  # Audio types: https://github.com/blakeblackshear/frigate/blob/dev/audio-labelmap.txt
  listen:
    - speech
    - scream
    - yell
    - gunshot
    - cat
    - purr
    - meow

detectors:
  cpu1:
    type: cpu
    num_threads: 3
  cpu2:
    type: cpu
    num_threads: 3

cameras:
  front_yard:
    zones:
      entire_yard:
        # Whole of our front yard: driveway, sidewalk and lawn
        coordinates: 0,720,1280,720,1280,396,730,278,0,364
        # coordinates: 0,1520,2688,1520,2688,836,1533,587,0,768
        objects:
          - person
          - dog
          - cat
      driveway:
        coordinates: 456,347,727,312,1203,417,1080,575
        # coordinates: 958,733,1527,659,2526,880,2268,1214
        objects:
          - car
      entrance:
        # Track the cat near the entrance
        coordinates: 847,673,949,720,1130,720,1208,522,1093,493
        # coordinates: 1779,1421,1993,1520,2373,1520,2537,1102,2295,1041
        objects:
          - cat
        filters:
            cat:
              min_score: 0.1
              threshold: 0.2
              min_area: 100
    record:
      events:
        required_zones:
          - entire_yard
          - driveway
    snapshots:
      required_zones:
        - entire_yard
        - driveway
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_yard
          input_args: preset-rtsp-restream
          roles:
            - audio
            - record
            - detect
      output_args:
        record: preset-record-generic-audio-copy
    audio:
      enabled: True
  back_yard:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/back_yard
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    zones:
      entire_yard:
        # Entire backyard
        coordinates: 0,720,1280,720,1280,59,546,71,0,296
        # coordinates: 0,1080,1920,1080,1920,88,819,106,0,444
        objects:
          - person
          - dog
          - cat
      entrance:
        # Track a cat near the entrance
        coordinates: 0,509,227,697,350,596,54,360,0,393
        # coordinates: 0,764,340,1046,525,894,81,540,0,590
        objects:
          - cat
        filters:
          cat:
            min_score: 0.1
            threshold: 0.2
            min_area: 100
    audio:
      enabled: False
    record:
      events:
        required_zones:
          - entire_yard
    snapshots:
      required_zones:
        - entire_yard

go2rtc:
  streams:
    back_yard:
      - "ffmpeg:rtsp://admin:xxx@cctv-back.xxx.ca/cam/realmonitor?channel=1&subtype=0"
      # Backyard camera has no audio, and does not work with go2rtc without the ffmpeg prefix, for unknown reason.
    front_yard:
      - "rtsp://admin:xxx@cctv-front.xxx.ca/cam/realmonitor?channel=1&subtype=0"
      - "ffmpeg:front_yard#audio=aac"

snapshots:
  enabled: True
  retain:
    default: 10

record:
  enabled: True
  events:
    retain:
      default: 10

objects:
  track:
    - person
    - dog
    - cat
    - car
  filters:
    dog:
      min_score: 0.2
      threshold: 0.3
    cat:
      min_score: 0.2
      threshold: 0.3
    person:
      min_area: 800
      min_score: 0.3
      threshold: 0.5
    car:
      min_area: 2000
      min_score: 0.3
      threshold: 0.5

detect:
  # width: 1280
  # height: 720
  stationary:
    max_frames:
      objects:
        car: 1000

Relevant log output

N/A

Frigate stats

{"cameras":{"back_yard":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.0,"capture_pid":657,"detection_enabled":1,"detection_fps":8.1,"ffmpeg_pid":662,"pid":651,"process_fps":1.3,"skipped_fps":3.7},"front_yard":{"audio_dBFS":-48.4962,"audio_rms":123.2079,"camera_fps":5.1,"capture_pid":652,"detection_enabled":1,"detection_fps":8.5,"ffmpeg_pid":660,"pid":650,"process_fps":1.0,"skipped_fps":4.2}},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"15":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"17":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"24":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"25":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"26":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"28":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"30":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"31":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"32":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"41":{"cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"42":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"80":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"81":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"82":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"89":{"cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"5.1","cpu_average":"0","mem":"0.1"},"97":{"cmdline":"python3 -u -m frigate","cpu":"9.6","cpu_average":"20","mem":"2.2"},"98":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"115":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.1"},"143":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"146":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.1"},"147":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"148":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"152":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"160":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"177":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"190":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"216":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"240":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"265":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"300":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"331":{"cmdline":"nginx: cache manager process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"351":{"cmdline":"nginx: cache loader process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"585":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.7"},"614":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"},"627":{"cmdline":"frigate.recording_manager","cpu":"1.4","cpu_average":"0","mem":"0.9"},"635":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(47)","cpu":"0.1","cpu_average":"0","mem":"0.1"},"637":{"cmdline":"frigate.detector.cpu1","cpu":"273.4","cpu_average":"173","mem":"0.8"},"639":{"cmdline":"frigate.detector.cpu2","cpu":"272.2","cpu_average":"173","mem":"0.8"},"641":{"cmdline":"frigate.output       ","cpu":"0.8","cpu_average":"0","mem":"0.8"},"644":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 2688x1520 -i pipe: -f mpegts -s 1273x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"645":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"650":{"cmdline":"frigate.process:front_yard","cpu":"28.6","cpu_average":"13","mem":"0.9"},"651":{"cmdline":"frigate.process:back_yard","cpu":"26.9","cpu_average":"13","mem":"1.0"},"652":{"cmdline":"frigate.capture:front_yard","cpu":"9.4","cpu_average":"4","mem":"0.8"},"657":{"cmdline":"frigate.capture:back_yard","cpu":"6.9","cpu_average":"0","mem":"0.8"},"660":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.13.0-65e3e67 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_yard -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy /tmp/cache/front_yard-%Y%m%d%H%M%S.mp4 -r 5 -vf fps=5,scale=2688:1520 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"38.6","cpu_average":"36","mem":"0.5"},"661":{"cmdline":"frigate.audio_manager","cpu":"1.5","cpu_average":"0","mem":"0.8"},"662":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.13.0-65e3e67 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/back_yard -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/back_yard-%Y%m%d%H%M%S.mp4 -r 5 -vf fps=5,scale=1920:1080 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"15.3","cpu_average":"9","mem":"0.3"},"668":{"cmdline":"ffmpeg -hide_banner -v error -allowed_media_types video+audio -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://admin:xxx@cctv-back.xxx.ca/cam/realmonitor?channel=1&subtype=0 -c copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/6c10da1c840f3ad5c12cb65584c8fccd","cpu":"1.5","cpu_average":"0","mem":"0.2"},"672":{"cmdline":"ffmpeg -vn -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.13.0-65e3e67 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_yard -f s16le -ar 16000 -ac 1 -y pipe:","cpu":"0.6","cpu_average":"0","mem":"0.2"},"679":{"cmdline":"ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://127.0.0.1:8554/front_yard?audio -c:a aac -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/97c350ea587379cd100e940f24c3d845","cpu":"4.1","cpu_average":"0","mem":"0.1"}},"detection_fps":16.6,"detectors":{"cpu1":{"detection_start":1699232561.524324,"inference_speed":121.85,"pid":637},"cpu2":{"detection_start":1699232561.489987,"inference_speed":129.47,"pid":639}},"processes":{"audioDetector":{"pid":661},"go2rtc":{"pid":89},"logger":{"pid":585},"recording":{"pid":627}},"service":{"last_updated":1699232561,"latest_version":"0.12.1","storage":{"/dev/shm":{"free":987.3,"mount_type":"tmpfs","total":1024.0,"used":36.7},"/media/frigate/clips":{"free":1954922.0,"mount_type":"nfs4","total":2027540.0,"used":72618.0},"/media/frigate/recordings":{"free":1954922.0,"mount_type":"nfs4","total":2027540.0,"used":72618.0},"/tmp/cache":{"free":12681.4,"mount_type":"overlay","total":60306.0,"used":45137.0}},"temperatures":{},"uptime":21,"version":"0.13.0-65e3e67"}}

Operating system

Other

Install method

Docker CLI

Coral version

CPU (no coral)

Any other information that may be helpful

No response

NickM-27 commented 10 months ago

This is working as expected and was changed in 0.13, where in the 0.13 docs does it say it defaults to 1280x720?

haimgel commented 10 months ago

Got it, thank you for your response. This is mentioned in the "new" Frigate Configuration Reference:

detect:
  # Optional: width of the frame for the input with the detect role (default: shown below)
  width: 1280
  # Optional: height of the frame for the input with the detect role (default: shown below)
  height: 720

Also: I think this is a breaking change and needs to be mentioned in the release notes, in the "Breaking Changes" section.

Mincka commented 10 months ago

Indeed, it's a breaking change. Personally, I decided to update all my coordinates to the native resolution. I used ChatGPT to teach it to upscale or downscale from 1280x720 to AxB and it worked very efficiently. I updated dozens of masks and zones in 10 minutes. Hopefully, @NickM-27 said that the ultimate goal is to have relative coordinates.