search

jakowenko / double-take

Unified UI and API for processing and training images for facial recognition.
https://hub.docker.com/r/jakowenko/double-take
MIT License
1.24k stars 98 forks source link

[FEAT] Use Without Frigate #206

Open slyticoon opened 2 years ago

slyticoon commented 2 years ago

Is your feature request related to a problem? Please describe.

I currently use the Unifi NVR system with the Unifi Protect integration in Home assistant. I would like to stay with this NVR platform, but be able to use advanced facial and object detection through the convenience of the unified UI provided by DoubleTake.

Describe the solution you'd like

I would like to use commands, service calls, or mqtt posts to send snapshot to doubletake. I would like to be able to create a snapshot manually without frigate, and receive the result.

Additional context

Perhaps this is already possible, and if so, please point me in the right direction.

jakowenko commented 2 years ago

Hey @slyticoon, thanks for checking the project out. Have you looked at the API docs? Depending on what your NVR can do, there is a /recognize endpoint that is exposed where you can pass an JPG image URL to it.

Does Unifi NVR post to a specific MQTT topic? I could build in functionality to listen to that as well depending on what the message is.

jakowenko commented 2 years ago

Also, checkout the cameras section of the docs. There is a way to configure a camera to listen to a specific MQTT topic assuming it has a jpeg encoded image.

https://github.com/jakowenko/double-take#cameras

cameras:
  front-door:
     snapshot:
       # process any jpeg encoded mqtt topic for facial recognition
       topic: path/to/topic
coreywagehoft commented 2 years ago

Is your feature request related to a problem? Please describe.

I currently use the Unifi NVR system with the Unifi Protect integration in Home assistant. I would like to stay with this NVR platform, but be able to use advanced facial and object detection through the convenience of the unified UI provided by DoubleTake.

Describe the solution you'd like

I would like to use commands, service calls, or mqtt posts to send snapshot to doubletake. I would like to be able to create a snapshot manually without frigate, and receive the result.

Additional context

Perhaps this is already possible, and if so, please point me in the right direction.

I am using a UniFi NVR with Frigate + Doubletake through HomeAssistant. The key is to have Frigate consume the RTSP streams from the NVR. You will need to configure a camera in Frigate from each camera you want to consume from UniFi. If possible use a Google Coral with frigate for extremely fast person detection.

upais commented 2 years ago

Also interesting is the use without Frigate, for example, through snapshot from the camera. I need to identify the face from the doorbell and the Frigate functionality is redundant for this. I saw the setting:

      snapshot:
        url:

But so far it has not been possible to do it, and it is not very convenient.

p.s. Thanks for a great product.

slyticoon commented 2 years ago

The benefit of going the mqtt route is that Unifi cameras already have object detection. So this could offload the resources required to process the presence of a person.

Useful in our current coral shortage.

On Mon, Jun 27, 2022, 17:12 Jesse Forkum @.***> wrote:

Hey Jakowenko,

Sorry for the delay on reply.

The NVR itself doesn't publish to mqtt, however it should be possible to use home assistant to publish an image from a unifi camera to a topic.

I went ahead and set up a frigate install to act as an intermediary between doubletake and unifi protect, but I'll try the mqtt solution, too.

And than you for putting your time toward developing a very awesome project.

On Mon, May 23, 2022, 13:50 David Jakowenko @.***> wrote:

Also, checkout the cameras section of the docs. There is a way to configure a camera to listen to a specific MQTT topic assuming it has a jpeg encoded image.

https://github.com/jakowenko/double-take#cameras

cameras: front-door: snapshot:

process any jpeg encoded mqtt topic for facial recognition

   topic: path/to/topic

— Reply to this email directly, view it on GitHub https://github.com/jakowenko/double-take/issues/206#issuecomment-1135023145, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQC47Q5TQNI3R4TUGMN7HMTVLPHPNANCNFSM5VL46DTA . You are receiving this because you were mentioned.Message ID: @.***>

upais commented 2 years ago

I have a Dahua doorbell, there is no object recognition function and does not publish in the mqtt. And HA does not have the functionality for publishing an image in a mqtt, I had to install Frigate.

upais commented 2 years ago

I have a Dahua doorbell, there is no object recognition function and does not publish in the mqtt. And HA does not have the functionality for publishing an image in a mqtt, I had to install Frigate.

Solved my issue by publishing the image via NodeRed.

LordNex commented 2 years ago

I have a Dahua doorbell, there is no object recognition function and does not publish in the mqtt. And HA does not have the functionality for publishing an image in a mqtt, I had to install Frigate.

Solved my issue by publishing the image via NodeRed.

Can you share your node flow?

upais commented 2 years ago

Can you share your node flow?

flows.zip

jjvelar commented 2 years ago

Hey @slyticoon, thanks for checking the project out. Have you looked at the API docs? Depending on what your NVR can do, there is a /recognize endpoint that is exposed where you can pass an JPG image URL to it.

Does Unifi NVR post to a specific MQTT topic? I could build in functionality to listen to that as well depending on what the message is.

Hi @jakowenko, I have created an additional camera with a mqtt topic but I can't see or select this new camera in "matches" screen. Any advice? Thanks!

jjvelar commented 2 years ago

I have a Dahua doorbell, there is no object recognition function and does not publish in the mqtt. And HA does not have the functionality for publishing an image in a mqtt, I had to install Frigate.

Solved my issue by publishing the image via NodeRed.

Hi @IvanUpa, thanks for sharing your flow. Could you also share how did you configured the mqtt camera in double-take? Can you see this mqtt camera in "matches" screen? Thanks

upais commented 2 years ago

Hi @IvanUpa, thanks for sharing your flow. Could you also share how did you configured the mqtt camera in double-take? Can you see this mqtt camera in "matches" screen? Thanks

Yes, I set it up according to the documentation.

cameras:
   doorbell:
     snapshot:
       topic: double-take/snapshot/doorbell
jjvelar commented 2 years ago

Hi @IvanUpa, thanks for sharing your flow. Could you also share how did you configured the mqtt camera in double-take? Can you see this mqtt camera in "matches" screen? Thanks

Yes, I set it up according to the documentation.

cameras:
   doorbell:
     snapshot:
       topic: double-take/snapshot/doorbell

Then I don't understand why I can't see the camera in "match" screen. That's my configuration:

cameras: doorbell_test: snapshot: topic: double-take/snapshot/doorbell-test flows (1).zip

and that's double take log output

22-08-29 08:21:17 info: processing doorbell_test: 547150fc-31ef-4efd-a79f-26cf48758ddc 22-08-29 08:21:18 info: done processing doorbell_test: 547150fc-31ef-4efd-a79f-26cf48758ddc in 0.31 sec 22-08-29 08:21:18 info: { id: '547150fc-31ef-4efd-a79f-26cf48758ddc', duration: 0.31, timestamp: '2022-08-29T06:21:18.127Z', attempts: 1, camera: 'doorbell_test', zones: [], matches: [], misses: [] }

any advice?

upais commented 2 years ago

Hi @IvanUpa, thanks for sharing your flow. Could you also share how did you configured the mqtt camera in double-take? Can you see this mqtt camera in "matches" screen? Thanks

Yes, I set it up according to the documentation.

cameras:
   doorbell:
     snapshot:
       topic: double-take/snapshot/doorbell

Then I don't understand why I can't see the camera in "match" screen. That's my configuration:

cameras: doorbell_test: snapshot: topic: double-take/snapshot/doorbell-test flows (1).zip

and that's double take log output

22-08-29 08:21:17 info: processing doorbell_test: 547150fc-31ef-4efd-a79f-26cf48758ddc 22-08-29 08:21:18 info: done processing doorbell_test: 547150fc-31ef-4efd-a79f-26cf48758ddc in 0.31 sec 22-08-29 08:21:18 info: { id: '547150fc-31ef-4efd-a79f-26cf48758ddc', duration: 0.31, timestamp: '2022-08-29T06:21:18.127Z', attempts: 1, camera: 'doorbell_test', zones: [], matches: [], misses: [] }

any advice?

What do you expect to see there and after what actions? Show full config double-take.

jjvelar commented 2 years ago

it is working now. I commented the frigate section in config, so no cameras from frigate were available. doorbell appeared and when I uncommented frigate section, all cameras -including doorbell one- appeared in "match" screen.

slyticoon commented 2 years ago

I guess what would be neat is if we could check a directory like "detections" for changes. Any new files are processed and the result is published on the mqtt topic.

That said, the mqtt method is basically that, but through a broker instead of file changes, and it avoids any docker container isolation issues.

On Mon, Jun 27, 2022, 17:14 Jesse Forkum @.***> wrote:

The benefit of going the mqtt route is that Unifi cameras already have object detection. So this could offload the resources required to process the presence of a person.

Useful in our current coral shortage.

On Mon, Jun 27, 2022, 17:12 Jesse Forkum @.***> wrote:

Hey Jakowenko,

Sorry for the delay on reply.

The NVR itself doesn't publish to mqtt, however it should be possible to use home assistant to publish an image from a unifi camera to a topic.

I went ahead and set up a frigate install to act as an intermediary between doubletake and unifi protect, but I'll try the mqtt solution, too.

And than you for putting your time toward developing a very awesome project.

On Mon, May 23, 2022, 13:50 David Jakowenko @.***> wrote:

Also, checkout the cameras section of the docs. There is a way to configure a camera to listen to a specific MQTT topic assuming it has a jpeg encoded image.

https://github.com/jakowenko/double-take#cameras

cameras: front-door: snapshot:

process any jpeg encoded mqtt topic for facial recognition

   topic: path/to/topic

— Reply to this email directly, view it on GitHub https://github.com/jakowenko/double-take/issues/206#issuecomment-1135023145, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQC47Q5TQNI3R4TUGMN7HMTVLPHPNANCNFSM5VL46DTA . You are receiving this because you were mentioned.Message ID: @.***>

slyticoon commented 2 years ago

Hey Jakowenko,

Sorry for the delay on reply.

The NVR itself doesn't publish to mqtt, however it should be possible to use home assistant to publish an image from a unifi camera to a topic.

I went ahead and set up a frigate install to act as an intermediary between doubletake and unifi protect, but I'll try the mqtt solution, too.

And than you for putting your time toward developing a very awesome project.

On Mon, May 23, 2022, 13:50 David Jakowenko @.***> wrote:

Also, checkout the cameras section of the docs. There is a way to configure a camera to listen to a specific MQTT topic assuming it has a jpeg encoded image.

https://github.com/jakowenko/double-take#cameras

cameras: front-door: snapshot:

process any jpeg encoded mqtt topic for facial recognition

   topic: path/to/topic

— Reply to this email directly, view it on GitHub https://github.com/jakowenko/double-take/issues/206#issuecomment-1135023145, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQC47Q5TQNI3R4TUGMN7HMTVLPHPNANCNFSM5VL46DTA . You are receiving this because you were mentioned.Message ID: @.***>

stalakerob commented 2 years ago

@slyticoon As jakowenko said, you can use the double-take API with the /recognize endpoint to do that. Here are the steps I used successfully:

  1. install a web server on the HA platform, e.g. apache2
  2. create a new directory under /var/www/html called snapshots
  3. put the image you want to be processed into the snapshots directory
  4. you can use curl to call the http endpoint: curl -k "http://111.222.333.444:3000/api/recognize?url=http://aaa.bbb.ccc.ddd:eeee/snapshots/cam_img.jpg&attempts=1&camera=test" where 111.222.333.444 is the host that runs double-take and aaa.bbb.ccc.ddd:eeee is the host and port that your web server runs on. So double-take will download the image from your web server, process it and return the result in json format. If configured it will also store the image in the matches directory. Note for camera=test you can put anything for the camera name. This name will be returned as part of the json result so that you can process it further if you like.

{"id":"967c2382-e565-47db-931d-ef600b86614a","duration":2.82,"timestamp":"2022-11-01T16:06:37.883Z","attempts":1,"camera":"test","zones":[],"counts":{"person":1,"match":1,"miss":0,"unknown":0},"matches":[{"name":"robert","confidence":100,"match":true,"box":{"top":758,"left":1045,"width":215,"height":215},"type":"manual","duration":2.75,"detector":"facebox","filename":"9e17450f-fe71-4d0b-a077-1c55ab8e15e4.jpg","base64":null}],"misses":[],"unknowns":[]}

Hope this helps.