search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.83k stars 1.71k forks source link

Issue with masks not matching image after saving them #10345

Closed simondsmason closed 7 months ago

simondsmason commented 7 months ago

I think I am having a similar issue to this one (https://github.com/blakeblackshear/frigate/discussions/9844). This started a few weeks ago - I originally noticed because my HASS notifications were not firing anymore. During this time period I did have to do quite a bit of work on my unraid as a cache drive failed but I am fully backed up so the config did (should) not have changed.

What is most noticeable is that before when I went into the Mask and Zone creator the image used to be twice the size. And the mask as it appears in the examples below was within the image boundaries. Now the image has shrunk. I have not tried re-saving the motion masks etc. since this happened. First I wanted to see what was going on before I did that. Thanks.

Here are some examples.

image image

Config:

mqtt:
  host: 192.168.2.110
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: frigate
  password: ###########6667
  stats_interval: 60
ffmpeg:
  hwaccel_args: preset-nvidia-h264
detectors:
  coral:
    type: edgetpu
    device: usb
record:
  enabled: true
  retain:
    days: 30
    mode: all
birdseye:
  enabled: True
  mode: continuous
  restream: True
rtmp:
  enabled: False
#  width: 1920
#  height: 1080
#  quality: 4
go2rtc:
  streams:
    driveway:
      - rtsp://admin:###########@192.168.2.234:554//h264Preview_01_main
      - "ffmpeg:driveway#audio=opus"
    drivewaysub:
      - rtsp://admin:###########@192.168.2.234:554//h264Preview_01_sub
      - "ffmpeg:driveway#audio=opus"
    back_garden:
      - rtsp://admin:###########@192.168.2.33:554//h264Preview_01_main
      - "ffmpeg:back_garden#audio=opus"
    back_gardensub:
      - rtsp://admin:###########@192.168.2.33:554//h264Preview_01_sub
      - "ffmpeg:back_garden#audio=opus"
    front_door:
      - rtsp://admin:###########@192.168.2.160:554//h264Preview_01_main
      - "ffmpeg:front_door#audio=opus"
    front_doorsub:
      - rtsp://admin:###########@192.168.2.160:554//h264Preview_01_sub
      - "ffmpeg:front_door#audio=opus"
    porch:
      - rtsp://admin:###########@192.168.2.161:554//h264Preview_01_main
      - "ffmpeg:porch#audio=opus"
    porchsub:
      - rtsp://admin:###########@192.168.2.161:554//h264Preview_01_sub
      - "ffmpeg:porch#audio=opus"
    front_garden:
      - rtsp://admin:###########@192.168.2.149:554//h264Preview_01_main
      - "ffmpeg:front_garden#audio=opus"
    front_gardensub:
      - rtsp://admin:###########@192.168.2.149:554//h264Preview_01_sub
      - "ffmpeg:front_garden#audio=opus"
#    kitchen:
#      - rtsp://admin:###########@192.168.2.227:554//h264Preview_01_main
#      - "ffmpeg:kitchen#audio=opus"
#    kitchensub:
#      - rtsp://admin:###########@192.168.2.227:554//h264Preview_01_sub
#      - "ffmpeg:kitchen#audio=opus"
    garage:
      - rtsp://192.168.2.110:8556/garage_inside
      - "ffmpeg:garage#audio=opus"
#    home_office:
#      - rtsp://admin:###########@192.168.2.217:554/live
#      - "ffmpeg:home_office#audio=opus"
#    home_theater:
#      - rtsp://admin:###########@192.168.2.233:554/live
#      - "ffmpeg:home_theater#audio=opus"
    birdseye:
      - rtsp://127.0.0.1:8554/birdseye
  webrtc:
    candidates:
      - 192.168.2.110:8555
      - stun:8555
cameras:
  driveway:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/drivewaysub?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - detect
##    detect:
##      width: 1280
##      height: 720            
    objects:
      track:
        - person
        - car
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    motion:
      mask:
        - 0,720,441,720,632,283,806,179,795,106,950,105,981,217,1060,293,1280,415,1280,0,513,0,0,0
  back_garden:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/back_garden?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/back_gardensub?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - detect
#    detect:
#      width: 1280
#      height: 720            
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    motion:
      mask:
        - 441,707,437,671,11,676,11,710
        - 910,371,904,512,939,562,1046,557,1085,411,1083,307,1044,246,974,240,929,281
    zones:
      back_step:
#        coordinates: 979,720,1122,720,1086,564,1059,533,965,533
        coordinates: 895,720,1122,720,1086,564,1059,533,893,536
  front_door:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
#        - path: rtsp://127.0.0.1:8554/front_doorsub?video=copy&audio=aac
#          input_args: preset-rtsp-restream
#          roles:
            - detect
    detect:
      width: 1280
      height: 720            
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: false
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 2
        post_capture: 2
    motion:
      mask:
        - 14,676,437,672,439,704,16,703
    zones:
      front_step:
        coordinates: 0,361,244,225,664,720,253,720
  porch:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/porch?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/porchsub?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - detect
#    detect:
#      width: 1280
#      height: 720            
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    motion:
      mask:
       - 442,707,442,664,14,665,12,711
  front_garden:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/front_garden?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/front_gardensub?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - detect
#    detect:
#      width: 1280
#      height: 720            
    objects:
      track:
        - person
        - dog
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    motion:
      mask:
        - 12,705,439,703,437,666,11,671
        - 0,0,1280,0,1280,274,908,292,351,354,0,452  
  kitchen:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/kitchen?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - rtmp
            - record
        - path: rtsp://127.0.0.1:8554/kitchensub?video=copy&audio=aac
          input_args: preset-rtsp-restream
          roles:
            - detect
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 2
        post_capture: 2
    detect:
      enabled: true
    motion:
      mask:
        - 0,720,433,720,433,677,0,676
        - 856,123,1193,125,1222,505,1193,720,700,720,735,530,851,521,854,315,730,306,732,141
  garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage?video=copy&audio=aac
          roles:
            - rtmp
            - detect
    detect:
      enabled: true
    birdseye:
      enabled: true
    motion:
      mask:
        - 771,720,1280,720,1280,669,769,668
  home_office:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/home_office?video=copy&audio=aac
          roles:
            - rtmp
            - detect
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    birdseye:
      enabled: true
    motion:
      mask:
        - 918,718,1251,716,1253,664,915,669
  home_theater:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/home_theater?video=copy&audio=aac
          roles:
            - rtmp
            - detect
    objects:
      track:
        - person
        - dog
        - cat
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 30
    record:
      enabled: true
      retain:
        days: 30
        mode: all
      events:
        pre_capture: 1
        post_capture: 1
    detect:
      enabled: false
    birdseye:
      enabled: True
    motion:
      mask:
        - 920,712,1255,712,1253,666,916,666
ui:
  live_mode: webrtc
#logger:
#  logs:
#    frigate.record: debug

Originally posted by @simondsmason in https://github.com/blakeblackshear/frigate/discussions/9844#discussioncomment-8700208

xconverge commented 7 months ago

Did you make the mask on the non-substream and then switch to using the substream? The mask needs to be created for the stream that will be used for detect. If you adjust resolution of this stream for any reason, the mask will need adjustment

nwish commented 7 months ago

I reported this a while ago and Nick said that they were going to make zone points relative to the camera view rather than hard coordinates in a later release. Until then, if you change the size of your detect feed, you'll want to comment out, or remove your existing zones.

simondsmason commented 7 months ago

I haven’t changed the config. But it gets weirder. I redid the masks and zones on the reduced image size. I restarted frigate and this time when I went into the mask and zone creator it now shows the full size image again. And the masks and zones are correct.

xconverge commented 7 months ago

If it were me, I would hardcode the detect resolution to the resolution of the substream. Your back garden camera is using the substream so it should be "the smaller" image probably, and the mask should be made/updated once this is done. What is the resolution of your substream?

Maybe the auto-detecting of the resolution is not working well/deterministically with your stream source, not sure really

NickM-27 commented 7 months ago

closing this as a duplicate of https://github.com/blakeblackshear/frigate/issues/6309 as that is what will fix it

NickM-27 commented 7 months ago

@xconverge is correct, I would definitely recommend manually setting detect resolution