frigate smart detections

[tsspmq]

Not sure why this happens

Setup:

Frigate - Bare metal docker container on the same machine the simple proxy and the unifi-cam-proxy run on.
UNVR with 1.18.1 MQTT in the home assistant supervised (A VM), authentication disabled.

Whichever camera gets the first smart detection, it finishes the first one. If a second camera gets a detection before first is done the first never finishes until the proxy is restarted even though frigate has moved on. The second camera (front_yard) never fires. If no other camera gets a detection the first one finishes fine. However, no further detections show up in protect - from any camera.

A proxy restart of the camera first to detect will allow it to grab another detection, once. Still, only the first camera to get one works and no others follow through.

Frigate keeps on trucking along, showing the detections and Home assistant keeps on getting its updates as well.

  | front_porch | person | 82.42% |   | 8/22/2021 | 12:24:09 PM | 12:24:24 PM   | front_yard | person | 81.64% |   | 8/22/2021 | 12:23:42 PM | 12:23:56 PM   | front_porch | person | 83.01% |   | 8/22/2021 | 12:23:18 PM | 12:23:29 PM   | front_porch | person | 80.47% |   | 8/22/2021 | 12:18:01 PM | 12:18:11 PM

Only the bottom one shows up in protect until that camera is restarted then I can get another one. The second and third event on same camera do not show up. The front_yard one never showed up. If I were to disable the porch and run to the yard one, it will fire - once until it restarts.

I thought this could have been due to the same source IP on the network sending the smart detections and it getting pissy inside protect, so I moved front_porch to a different machine on the same subnet as the DVR.

Same thing happens, front_porch gets the first detection, it works. The second one (I have to walk past first to get to second) never fires. If I stop the first proxy and restart the second proxy, and walk to the second camera it will fire.

Now, all the camera logs show it seeing the start and finish of the events via MQTT, its just not getting this across to protect.

This could just be protect buggy, but I dont have any unifi cams to test if it works fine with two cameras, and if a second fires while first is still in an event.

[tsspmq]

One thing I haven't tried is two different client certificates. I use the same one for all 16 cameras (And all 16 work fine all day long).

I'll test that to see if it matters by generating a new one on the second computer and just running those two cams.

[keshavdv]

I've never tested with more than one Frigate source, so it's possible the implementation doesn't handle this well. Can you share the output of two unifi-cam-proxy instances with --verbose enabled as you walk past one and then the other triggering motion events? It's possible that the motion event IDs which always just start from 0 and increment need to be unique even across cameras.

[tsspmq]

I think this is an issue with my MQTT broker.

Might be due to it being in home assistant's docker stack.

Socket error on client auto-54D24E3C-88FE-2976-4C89-F577AF38C6D4, disconnecting.

I'll setup a broker outside and see if the issue stops, I dont need it in that stack. This may be tomorrow though.

[tsspmq]

Hey I got around to fiddling with this some today to get logs, attached at end.

I think this may be part of the issue:

This is in frigate/events:

"Message published for each changed event. The first message is published when the tracked object is no longer marked as a false_positive. When frigate finds a better snapshot of the tracked object or when a zone change occurs, it will publish a message with the same id. When the event ends, a final message is published with end_time set."

I wonder if tracking the message id (in the situation where two events happen on the same camera back to back or a car comes by while a persons is there) and adding in a check for "update" type messages in addition to "new and end" may help with this.

Not sure how much of a challenge this is in python.

I've had it get stuck in "ongoing" with just one camera running and it never sees the end message.

I dont know if this update message needs to be handled or not. I also see MQTT messages with false positive in them, and I'm not sure you're factoring in the false positive or not. I assumed it only pushed a message when it was "sure" it was an object.

camera6.txt camera3.txt

Example, this is a parked vehicle and I havent removed that object from the camera since well, its going to always see that car:

{"before": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746627.991348, "label": "car", "top_score": 0.0, "false_positive": true, "start_time": 1629746627.991348, "end_time": null, "score": 0.82421875, "box": [126, 122, 758, 478], "area": 224992, "region": [0, 0, 984, 984], "current_zones": [], "entered_zones": [], "thumbnail": null}, "after": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746629.982454, "label": "car", "top_score": 0.734375, "false_positive": false, "start_time": 1629746627.991348, "end_time": null, "score": 0.734375, "box": [136, 123, 620, 489], "area": 177144, "region": [11, 80, 627, 696], "current_zones": [], "entered_zones": [], "thumbnail": null}, "type": "new"}

{"before": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746629.982454, "label": "car", "top_score": 0.734375, "false_positive": false, "start_time": 1629746627.991348, "end_time": null, "score": 0.734375, "box": [136, 123, 620, 489], "area": 177144, "region": [11, 80, 627, 696], "current_zones": [], "entered_zones": [], "thumbnail": null}, "after": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746633.77056, "label": "car", "top_score": 0.77734375, "false_positive": false, "start_time": 1629746627.991348, "end_time": null, "score": 0.51171875, "box": [396, 602, 597, 717], "area": 23115, "region": [0, 32, 688, 720], "current_zones": [], "entered_zones": [], "thumbnail": null}, "type": "update"}

{"before": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746633.77056, "label": "car", "top_score": 0.77734375, "false_positive": false, "start_time": 1629746627.991348, "end_time": null, "score": 0.51171875, "box": [396, 602, 597, 717], "area": 23115, "region": [0, 32, 688, 720], "current_zones": [], "entered_zones": [], "thumbnail": null}, "after": {"id": "1629746627.991348-ob3ktl", "camera": "driveway_corner", "frame_time": 1629746633.77056, "label": "car", "top_score": 0.77734375, "false_positive": false, "start_time": 1629746627.991348, "end_time": 1629746640.742344, "score": 0.51171875, "box": [396, 602, 597, 717], "area": 23115, "region": [0, 32, 688, 720], "current_zones": [], "entered_zones": [], "thumbnail": null}, "type": "end"}

[keshavdv]

I think this is fixed by 2351742ff899, can you try it out and see if that helps?

[samuelhautcoeur]

I think this is fixed by 2351742, can you try it out and see if that helps?

You beat me to it, I just came with #79

[tsspmq]

appears to be working now. I've got two cameras firing a car detection over and over (like they should) because there is a car in the view. I need to turn off car detection for them but that shows its allowing two to detect without pissing the other off

[tsspmq]

7 Hours and all 8 frigate cameras chugging along fine. Great work.

[jkblm]

@tsspmq I have unifi-cam-proxy running well on RBPi4B/4GB (running current Rasberry Pi O/S) with 5 Hikvision cameras in Unifi UNVR Protect 1.9.0

I am not able to successfully install Frigate/MQTT installed on the same RBPi with unifi-cam-proxy.

How did you: "Bare metal docker container install Frigate/MQTT on the same machine the simple proxy and the unifi-cam-proxy run on."

Thanks in advance!

PS: I am able to get Frigate/MQTT installed and connected to the same cameras running at the same times as above.. This is on a second RPi4B with Home Assistant as the base O/S. But I have not been successful in installing unifi-cam-proxy on this machine. Goal is to have all working and integrated on one machine:

• unifi-cam-proxy
• Frigate/MQTT
• Unifi Protect 1.9.0 on UNVR connected and recording motion with the 5 (or more) Hikvision PTZ/Dome cameras.