blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
17.67k stars 1.62k forks source link

[Config Support]: Detection Staying In Progress #4345

Closed rayzorben closed 1 year ago

rayzorben commented 1 year ago

Describe the problem you are having

I did a walk around to test my setup. After returning, 3 of the 4 zones stayed In Progress, they started slowly clearing up but even now 2 minutes later it is still showing In Progress on one of the cameras

image

I thought maybe a hardware issue, but it looks OK

image

Version

0.11.1

Frigate config file

mqtt:
  host: homeassistant.local
  user: foo
  password: foo

ffmpeg:
  #hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/card0 -hwaccel_output_format yuv420p
  #hwaccel_args:
    #-hwaccel h264_rkmpp
    #-hwaccel_device /dev/dri/card0
    #-hwaccel_output_format yuv420p
  input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -stimeout 5000000 -use_wallclock_as_timestamps 1

detect:
  width: 704
  height: 480
  fps: 5

snapshots:
  enabled: True

objects:
  track:
    - car
    - person
    - dog

cameras:
  front_porch:
    motion:
      mask:
        - 0,0,0,74,70,52,173,32,274,29,416,33,530,57,653,92,704,85,704,0
    zones:
      front_street:
        coordinates: 139,39,163,31,206,36,247,23,327,25,416,34,515,51,592,72,644,89,674,97,658,172,634,163,566,138,443,112,286,76,144,86,142,66
        objects:
          - person
          - car
          - dog
      front_porch_area:
        coordinates: 0,480,0,76,140,41,140,82,258,86,430,111,652,172,704,90,704,480
        objects:
          - person
          - dog
    snapshots:
      required_zones:
        - front_street
        - front_porch_area
    ffmpeg:
      inputs:
        - path: rtsp://admin:foo@192.168.1.20:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
            - rtmp

  driveway:
    motion:
      mask:
        - 0,0,704,0,704,166,0,184
    ffmpeg:
      inputs:
        - path: rtsp://admin:foo@192.168.1.21:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
            - rtmp
    zones:
      front_street:
        coordinates: 0,250,704,239,704,170,0,181
        objects:
          - person
          - car
          - dog
      driveway_area:
        coordinates: 704,480,0,480,0,226,704,233
        objects:
          - person
          - car
          - dog
    snapshots:
      required_zones:
        - front_street
        - driveway_area

  south_side:
    motion:
      mask:
        - 0,0,0,56,38,45,114,42,345,76,629,156,704,238,704,0
    ffmpeg:
      inputs:
        - path: rtsp://admin:foo@192.168.1.22:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
            - rtmp

  back_north:
    motion:
      mask:
        - 626,173,630,156,629,134,543,110,449,97,316,69,171,67,78,63,0,79,0,0,704,0,704,203
    ffmpeg:
      inputs:
        - path: rtsp://admin:foo@192.168.1.23:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
            - rtmp

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-11-09 08:44:43] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21)
[2022-11-09 08:44:43] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-11-09 08:44:43] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-11-09 08:44:43] peewee_migrate                 INFO    : There is nothing to migrate
[2022-11-09 08:44:44] frigate.app                    INFO    : Output process started: 213
[2022-11-09 08:44:44] detector.cpu                   INFO    : Starting detection process: 211
[2022-11-09 08:44:44] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-11-09 08:44:44] ws4py                          INFO    : Using epoll
[2022-11-09 08:44:44] frigate.app                    INFO    : Camera processor started for front_porch: 217
[2022-11-09 08:44:44] frigate.app                    INFO    : Camera processor started for driveway: 219
[2022-11-09 08:44:44] frigate.app                    INFO    : Camera processor started for south_side: 220
[2022-11-09 08:44:44] frigate.app                    INFO    : Camera processor started for back_north: 222
[2022-11-09 08:44:44] frigate.app                    INFO    : Capture process started for front_porch: 223
[2022-11-09 08:44:44] frigate.app                    INFO    : Capture process started for driveway: 225
[2022-11-09 08:44:44] frigate.app                    INFO    : Capture process started for south_side: 231
[2022-11-09 08:44:44] frigate.app                    INFO    : Capture process started for back_north: 238
[2022-11-09 08:44:46] ws4py                          INFO    : Using epoll
[2022-11-09 08:44:52] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:36906]
[2022-11-09 08:44:53] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:36906]
[2022-11-09 08:44:54] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42028]
[2022-11-09 08:44:55] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42028]
[2022-11-09 08:44:55] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42042]
[2022-11-09 08:44:56] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42042]
[2022-11-09 08:44:56] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42048]
[2022-11-09 08:47:39] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:42048]
[2022-11-09 08:47:40] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39738]
[2022-11-09 08:47:48] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52878]
[2022-11-09 08:47:54] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52878]
[2022-11-09 08:47:55] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:46536]
[2022-11-09 08:47:56] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:46536]
[2022-11-09 08:48:00] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:46542]
[2022-11-09 08:48:04] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:46542]
[2022-11-09 08:48:32] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:41250]
[2022-11-09 08:48:36] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:41250]
[2022-11-09 08:48:37] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39738]
[2022-11-09 08:48:38] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:41392]
[2022-11-09 08:48:41] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:41392]
[2022-11-09 08:49:09] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:41470]
[2022-11-09 08:50:30] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:47892]
[2022-11-09 08:50:30] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:47892]
[2022-11-09 08:51:30] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:41470]
[2022-11-09 08:51:31] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:46784]
[2022-11-09 08:53:10] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:46784]

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Any other information that may be helpful

No response

rayzorben commented 1 year ago

So I went into debug mode and it seems to think that there is a 51% match a wall ornament is a person. I am guessing that this is keeping it in progress? If so, how can I reduce false positives on areas like this by ignoring say something under 70%?

image

NickM-27 commented 1 year ago

Most likely is a false positive, check the debug camera view with bounding boxes to see where the detected object is

rayzorben commented 1 year ago

Most likely is a false positive, check the debug camera view with bounding boxes to see where the detected object is

Just wanted to see I really appreciate how quickly you respond and the effort you put into this project to help out!

NickM-27 commented 1 year ago

You probably won't want to ignore scores. The reason this happens is because Frigate saw you (a high scoring person) and then you left but frigate saw this small object and it assumes it is still you.

The best way to avoid this one is probably to add a min_area filter. The number after the score is the area, no person would actually be that small so if you set a filter for person of 1000 it should stop those.

Just wanted to see I really appreciate how quickly you respond and the effort you put into this project to help out!

Thanks! 😄

ekt- commented 1 year ago

I am seeing such a thing and I don't think it has ever happened in the last year.

an event is stuck on 'in progress'. in the meantime other events have happened that have closed properly. the debug at the moment shows a stationary object (a car parked for days in the same place) at 51% if I click on the 'IN PROGRESS' event I see a 2 minute clip

the two things that have changed recently are that I installed 11.1 and now use Coral instead of cpu

NickM-27 commented 1 year ago

I am seeing such a thing and I don't think it has ever happened in the last year.

an event is stuck on 'in progress'. in the meantime other events have happened that have closed properly. the debug at the moment shows a stationary object (a car parked for days in the same place) at 51%

if I click on the 'IN PROGRESS' event I see a 2 minute clip

the two things that have changed recently are that I installed 11.1 and now use Coral instead of cpu

Nothing for stationary object tracking changed in 0.11. If a car is parked then it is 100% expected that the event would be in progress and depending on your config not all of that event would be kept as recordings

https://docs.frigate.video/configuration/record#what-do-the-different-retain-modes-mean

NGrey5 commented 1 year ago

I'm seeing the same thing on my end as well. I'm using the default detection settings for stationary objects. Before upgrading to version 11 I never had an "IN PROGRESS" for an event. Someone will enter frame, leave frame, and then it'll think one of my bushes is a person. Shouldn't it stop the detection or clip if the "bush-person" is stationary for a certain amount of time?

Another example, a car pulled into my driveway, parked, and the clip will be in progress for over an hour (I had to manually stop it by deleting it) when the car is not moving. The bounding boxes show no movement and there is no other motion detected in the frame.

It has also thought a couple of chairs were cars on my patio when those chairs have been in the same spot for over a year and I never had a detection go off for them previously. Very odd.

I have had no changes to my config or hardware. The only thing that changed was the upgrade from v10 to v11. I hope this can help with figuring out if there's a potential issue. Thank you!

NickM-27 commented 1 year ago

Shouldn't it stop the detection or clip if the "bush-person" is stationary for a certain amount of time?

No, there is no limit on how long an object can be stationary. The way to fix this would be filtering out the bush either with an object mask or size / ratio filter.

Another example, a car pulled into my driveway, parked, and the clip will be in progress for over an hour (I had to manually stop it by deleting it) when the car is not moving. The bounding boxes show no movement and there is no other motion detected in the frame.

This is the expected behavior and frigate working as it should. An event is valid until the object is no longer visible.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

gieljnssns commented 1 year ago

This has happened in my instance to.

gieljnssns commented 1 year ago

49A42D22-B865-4730-804F-FAF6CAE1FA5D F0726C8C-15CC-46FA-A424-577429C26202 1B6F0AB1-03B5-4C34-9931-988899CEB3B6

gieljnssns commented 1 year ago
{
  "birdseye": {
    "enabled": false,
    "height": 720,
    "mode": "objects",
    "quality": 8,
    "width": 1280
  },
  "cameras": {
    "tuin": {
      "best_image_timeout": 60,
      "birdseye": {
        "enabled": false,
        "mode": "objects"
      },
      "detect": {
        "enabled": true,
        "fps": 5,
        "height": 480,
        "max_disappeared": 25,
        "stationary": {
          "interval": 0,
          "max_frames": {
            "default": null,
            "objects": {}
          },
          "threshold": 50
        },
        "width": 640
      },
      "ffmpeg": {
        "global_args": "-hide_banner -loglevel warning",
        "hwaccel_args": "-hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p",
        "input_args": "-avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1",
        "inputs": [
          {
            "global_args": [],
            "hwaccel_args": [],
            "input_args": [],
            "path": "rtsp://admin:xxxxxxxxxx@192.168.89.132:554/cam/realmonitor?channel=1&subtype=1",
            "roles": [
              "detect"
            ]
          },
          {
            "global_args": [],
            "hwaccel_args": [],
            "input_args": [],
            "path": "rtsp://admin:cxccxxx@192.168.89.132:554/live",
            "roles": [
              "record"
            ]
          }
        ],
        "output_args": {
          "detect": "-f rawvideo -pix_fmt yuv420p",
          "record": "-f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an",
          "rtmp": [
            "-c",
            "copy",
            "-f",
            "flv"
          ]
        }
      },
      "ffmpeg_cmds": [
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://admin:xxxxxxxxccc@192.168.89.132:554/cam/realmonitor?channel=1&subtype=1 -r 5 -s 640x480 -f rawvideo -pix_fmt yuv420p pipe:",
          "roles": [
            "detect"
          ]
        },
        {
          "cmd": "ffmpeg -hide_banner -loglevel warning -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://admin:xxxxxxxxxxx@192.168.89.132:554/live -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an /tmp/cache/tuin-%Y%m%d%H%M%S.mp4",
          "roles": [
            "record"
          ]
        }
      ],
      "live": {
        "height": 720,
        "quality": 8
      },
      "motion": {
        "contour_area": 30,
        "delta_alpha": 0.2,
        "frame_alpha": 0.2,
        "frame_height": 50,
        "improve_contrast": true,
        "mask": [
          "0,313,239,313,246,419,430,410,417,0,0,0",
          "1280,720,855,720,830,0,1280,0",
          "427,316,540,286,540,446,672,442,662,234,885,221,874,0,420,0"
        ],
        "mqtt_off_delay": 30,
        "threshold": 25
      },
      "mqtt": {
        "bounding_box": false,
        "crop": true,
        "enabled": true,
        "height": 270,
        "quality": 70,
        "required_zones": [],
        "timestamp": true
      },
      "name": "tuin",
      "objects": {
        "filters": {
          "person": {
            "mask": [
              "0,313,239,313,246,419,430,410,417,0,0,0",
              "1280,720,855,720,830,0,1280,0",
              "427,316,540,286,540,446,672,442,662,234,885,221,874,0,420,0"
            ],
            "max_area": 24000000,
            "max_ratio": 24000000,
            "min_area": 5000,
            "min_ratio": 0,
            "min_score": 0.6,
            "threshold": 0.67
          }
        },
        "mask": "",
        "track": [
          "person"
        ]
      },
      "record": {
        "enabled": true,
        "events": {
          "objects": [
            "person"
          ],
          "post_capture": 10,
          "pre_capture": 10,
          "required_zones": [],
          "retain": {
            "default": 7,
            "mode": "motion",
            "objects": {
              "person": 10
            }
          }
        },
        "expire_interval": 60,
        "retain": {
          "days": 0,
          "mode": "motion"
        },
        "retain_days": null
      },
      "rtmp": {
        "enabled": false
      },
      "snapshots": {
        "bounding_box": true,
        "clean_copy": true,
        "crop": false,
        "enabled": true,
        "height": 175,
        "quality": 70,
        "required_zones": [],
        "retain": {
          "default": 10,
          "mode": "motion",
          "objects": {
            "person": 15
          }
        },
        "timestamp": true
      },
      "timestamp_style": {
        "color": {
          "blue": 255,
          "green": 255,
          "red": 255
        },
        "effect": null,
        "format": "%d/%m/%Y %H:%M:%S",
        "position": "tl",
        "thickness": 2
      },
      "ui": {
        "dashboard": true,
        "order": 0
      },
      "zones": {}
    }
  },
  "database": {
    "path": "/media/frigate/frigate.db"
  },
  "detect": {
    "enabled": true,
    "fps": 5,
    "height": 720,
    "max_disappeared": null,
    "stationary": {
      "interval": 0,
      "max_frames": {
        "default": null,
        "objects": {}
      },
      "threshold": null
    },
    "width": 1280
  },
  "detectors": {
    "coral": {
      "device": "usb",
      "num_threads": 3,
      "type": "edgetpu"
    }
  },
  "environment_vars": {},
  "ffmpeg": {
    "global_args": "-hide_banner -loglevel warning",
    "hwaccel_args": "-hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p",
    "input_args": "-avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1",
    "output_args": {
      "detect": "-f rawvideo -pix_fmt yuv420p",
      "record": "-f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an",
      "rtmp": [
        "-c",
        "copy",
        "-f",
        "flv"
      ]
    }
  },
  "live": {
    "height": 720,
    "quality": 8
  },
  "logger": {
    "default": "info",
    "logs": {}
  },
  "model": {
    "height": 320,
    "labelmap": {},
    "labelmap_path": null,
    "path": null,
    "width": 320
  },
  "motion": null,
  "mqtt": {
    "client_id": "frigate",
    "host": "core-mosquitto.local.hass.io",
    "password": "xxxxxxxx",
    "port": 1883,
    "stats_interval": 60,
    "tls_ca_certs": null,
    "tls_client_cert": null,
    "tls_client_key": null,
    "tls_insecure": null,
    "topic_prefix": "frigate",
    "user": "hass"
  },
  "objects": {
    "filters": null,
    "mask": "",
    "track": [
      "person"
    ]
  },
  "plus": {
    "enabled": false
  },
  "record": {
    "enabled": true,
    "events": {
      "objects": [
        "person"
      ],
      "post_capture": 10,
      "pre_capture": 10,
      "required_zones": [],
      "retain": {
        "default": 7,
        "mode": "motion",
        "objects": {
          "person": 10
        }
      }
    },
    "expire_interval": 60,
    "retain": {
      "days": 0,
      "mode": "motion"
    },
    "retain_days": null
  },
  "rtmp": {
    "enabled": false
  },
  "snapshots": {
    "bounding_box": true,
    "clean_copy": true,
    "crop": false,
    "enabled": true,
    "height": 175,
    "quality": 70,
    "required_zones": [],
    "retain": {
      "default": 10,
      "mode": "motion",
      "objects": {
        "person": 15
      }
    },
    "timestamp": true
  },
  "timestamp_style": {
    "color": {
      "blue": 255,
      "green": 255,
      "red": 255
    },
    "effect": null,
    "format": "%d/%m/%Y %H:%M:%S",
    "position": "tl",
    "thickness": 2
  },
  "ui": {
    "use_experimental": false
  }
}
NickM-27 commented 1 year ago

@gieljnssns this is due to your motion masks. You're making out areas where true objects are which causes them to get stuck. The docs explain that this is not how motion masks are meant to be used. https://docs.frigate.video/configuration/masks

rayzorben commented 1 year ago

You probably won't want to ignore scores. The reason this happens is because Frigate saw you (a high scoring person) and then you left but frigate saw this small object and it assumes it is still you.

The best way to avoid this one is probably to add a min_area filter. The number after the score is the area, no person would actually be that small so if you set a filter for person of 1000 it should stop those.

Just wanted to see I really appreciate how quickly you respond and the effort you put into this project to help out!

Thanks! 😄

@NickM-27 I didn't want to reopen this or start a new ticket, but your guidance was to set the min_area. I am still having issues with this. As you can see, the area here is reporting way too large, it seems unrealistic

image

NickM-27 commented 1 year ago

@rayzorben in that case probably use a person object mask on the upper part of the wall

rayzorben commented 1 year ago

I already have a Mask there, is it somehow different?

image

NickM-27 commented 1 year ago

I already have a Mask there, is it somehow different?

image

motion masks don't stop objects from being detected. Also, like zones, object masks are evaluated based on the bottom center pixel of the objects bounding box

euro2 commented 1 year ago

i also got one stuck. But the location is where i really want to see people early to toggle some lights in the garden. The size of the stuck box is exactly a person lol. So i cant edit that..

Screenshot 2023-02-06 at 12-28-42 Frigate – Home Assistant Screenshot 2023-02-06 at 12-28-28 Frigate – Home Assistant

blakeblackshear commented 1 year ago

@euro2 do you have any kind of motion mask in that area? You could try setting a max_frames for person in your stationary config as well.

euro2 commented 1 year ago

thank you. I have no motion mask there since i want to find a person there for some lights. I had a look at max frames. So it looks like its a kill switch for tracked stationary objects. Ill go and tinker with that.

Not sure if the max frames is based on the detection speed of 5fps in my case, or the actual stream fps which is 25 fps.

NickM-27 commented 1 year ago

Not sure if the max frames is based on the detection speed of 5fps in my case, or the actual stream fps which is 25 fps.

It is based on detect fps

euro2 commented 1 year ago

Not sure if the max frames is based on the detection speed of 5fps in my case, or the actual stream fps which is 25 fps.

It is based on detect fps

Thank you. I wont ever have any persons stay still on that spot forever :) so max_frames will do the trick im sure!