search

jakowenko / double-take

Unified UI and API for processing and training images for facial recognition.
https://hub.docker.com/r/jakowenko/double-take
MIT License
1.25k stars 99 forks source link

[FR] Snapshot from Camera Directly #57

Closed Raphael909 closed 3 years ago

Raphael909 commented 3 years ago

Hello @jakowenko , Hope you doing well. What you think about getting snapshot from Camera snapshot URL directly instead of MQTT and Frigate. Most of the time people use sub-stream of the camera for object detection or say in Frigate and use High res stream for recording. Now, frigate decodes sub-stream and post low res snapshot to mqtt and this makes face recognition and also training less accurate.

So, I was like, what if we grab image directly from camera on say some trigger and let double-take use that for face recognition or for training. It could be like how frigate does.. Provide camera level configuration and honor that if URL is given or use frigate topic for snapshot. either or but certainly not both probably.

Benefits,

how you feel about this?

Thank you again for your time and effort!

jakowenko commented 3 years ago

Hey Raphael,

Thanks for the feature suggestion. I think that makes sense for cameras that support a snapshot, but what about cameras that just have a RTSP URL? Were you hoping you could configure cameras similar to how they are in Frigate and an ffmpeg process would turn that feed into an image to process?

Overall I think trying to decouple Double Take from other projects makes sense and allows people to use it in other ways.

Raphael909 commented 3 years ago

Hi, that is a very good point.. I was not thinking of ffmpeg at all instead I was more on the lines of such type of integration Camera ---> Surveillance system (BI)---> [Frigate] ---> HA

Config {"cameras":[{ "Driveway-camera":{ "name":"Driveway", "height":500, "priority":"mqtt", "snapshot":{ "mqtt":"<<some-topic>>", "url": "http://someURL/current/cgi.bin" } } }] } For HA integration , you already have code in place to do mqtt publish.

jakowenko commented 3 years ago

I don't think I'm fully understanding, maybe because I don't know the capabilities of Blue Iris. In your example with the camera URL being http://something/cgi/current.bin, what is the output of that? A jpg image or some sort of stream?

Are you picturing Blue Iris detecting motion then triggering something within Double Take to start processing the camera images?

Raphael909 commented 3 years ago

I was still trying to fix my config style :P . Sorry, yeah it will be jpg. I have bunch of camera in my system .. likes of Wyze with dafang hacks, Amcrest etc. and they all support current image url a,k,a snapshots but end of the day most cameras and most nvrs, hardware or software does support current image url

jakowenko commented 3 years ago

No problem, the edited config helped me better understand. What is the static URL of the Wyze image? I have some of those cameras but have been using the RTSP one currently. Just looking for something to test with.

Raphael909 commented 3 years ago

so, Wyze I have uses dafang and it provides image at :https://192.xxx.xx.119/cgi-bin/currentpic.cgi .. you may also include width and height as query param

jakowenko commented 3 years ago

Ah gotcha, so I need to flash them with the Dafang firmware before I can use that. I'll give it a go on one of them.

jakowenko commented 3 years ago

Can you curl that URL curl --head https://192.xxx.xx.119/cgi-bin/currentpic.cgi and see what the content type is?

Raphael909 commented 3 years ago

Are you picturing Blue Iris detecting motion then triggering something within Double Take to start processing the camera images?

Yes that what I had in my mind.. It could be MQTT controlled that double-take defines or via a GET

Raphael909 commented 3 years ago

Untitled

I dont have curl on my machine but I think this network grab might be helpful to you

jakowenko commented 3 years ago

Thank you! I'll probably start with a basic implementation and then we can try to expand from there.

Raphael909 commented 3 years ago

Ah gotcha, so I need to flash them with the Dafang firmware before I can use that. I'll give it a go on one of them.

here is the link .. just in case https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks.

jakowenko commented 3 years ago

@Raphael909 in the short term would just the /api/recognize endpoint work for this?

You could pass https://192.xxx.xx.119/cgi-bin/currentpic.cgi as the URL and increase the attempts param to be something higher than 1 to process that image multiple times.

Just an idea until I can build something that is defined via the config.

Query Params Default Description
url URL of image to pass to facial recognition detectors
attempts 1 Number of attempts before stopping without a match
results best Options: best, all
break true Break attempt loop if a match is found
camera double-take Camera name used in output results
Raphael909 commented 3 years ago

Good idea! . Probably, you will need time to think through, flush out design and some details.

jakowenko commented 3 years ago

@Raphael909, I just pushed a beta build with a basic implementation of this so far.

cameras:
  driveway:
    snapshot:
      url: https://192.xxx.xx.119/cgi-bin/currentpic.cgi

This can then be invoked via a GET request at - /api/cameras/driveway

I haven't written the MQTT logic for this yet, but I was thinking about automatically subscribing to that topic when the API starts. That way any message from that topic would be processed and the API could still be invoked via the GET request.

Raphael909 commented 3 years ago

@Raphael909, I just pushed a beta build with a basic implementation of this so far.

cameras:
  driveway:
    snapshot:
      url: https://192.xxx.xx.119/cgi-bin/currentpic.cgi

This can then be invoked via a GET request at - /api/cameras/driveway

I haven't written the MQTT logic for this yet, but I was thinking about automatically subscribing to that topic when the API starts. That way any message from that topic would be processed and the API could still be invoked via the GET request.

I will give this a spin tomorrow evening after work probably.. Good stuff man! I was also thinking, if you would like to create a post for double-take in this forum https://ipcamtalk.com/.. Lots and lots of IP camera / blue iris / ISPY enthusiast . What you have started here is a fantastic project!

jakowenko commented 3 years ago

@Raphael909, thank you! Is there a specific sub forum I should post in? I'd love to get some of the feedback from their users.

tunip commented 3 years ago

@Raphael909, I just pushed a beta build with a basic implementation of this so far.

cameras:
  driveway:
    snapshot:
      url: https://192.xxx.xx.119/cgi-bin/currentpic.cgi

This can then be invoked via a GET request at - /api/cameras/driveway

I haven't written the MQTT logic for this yet, but I was thinking about automatically subscribing to that topic when the API starts. That way any message from that topic would be processed and the API could still be invoked via the GET request.

Tried this beta, but received a Cannot GET /api/cameras/eingang. My camera needs basic auth which I entered in the URL in the config.yml: url: http://<user>:<password>@<ip>...

jakowenko commented 3 years ago

Tried this beta, but received a Cannot GET /api/cameras/eingang. My camera needs basic auth which I entered in the URL in the config.yml: url: http://<user>:<password>@<ip>...

Does your config look like this?

cameras:
  eingang:
    snapshot:
      url: http://<user>:<password>@<ip>
tunip commented 3 years ago

exactly...

cameras:
  eingang:
    snapshot:
      url: http://username:password@192.168.5.19/cgi-bin/snapshot.cgi?Channel=0
jakowenko commented 3 years ago

@tunip Can you try to repull the image and recreate your container?

The route should at the very least output { error: "camera config not found" } if it can't find the camera configuration. The Cannot GET /api/cameras/eingang error leads me to think you don't have the latest beta image.

The query string in the snapshot URL may cause issues though, I'll need to debug that later. But the route should at least be working.

tunip commented 3 years ago

repulled the image and recreated the container.

I use the following docker-compose part to start double-take. Changed ":latest" to ":beta-0.7.0".

  double-take:
    image: jakowenko/double-take:beta-0.7.0
    restart: unless-stopped
    container_name: double-take
    environment:
      - TZ=Europe/Berlin
    volumes:
      - double-take:/double-take
      - double-take_data:/.storage

Fetching

$ docker-compose up -d
Pulling double-take (jakowenko/double-take:beta-0.7.0)...
beta-0.7.0: Pulling from jakowenko/double-take
Digest: sha256:b7d55a2ce782c4da91815b644e6098762e73e18fe84284395da43d42141c8351
Status: Downloaded newer image for jakowenko/double-take:beta-0.7.0
compreface-core is up-to-date
compreface-postgres-db is up-to-date
deepstack is up-to-date
Recreating double-take ... 
compreface-api is up-to-date
compreface-admin is up-to-date
Recreating double-take ... done

"docker images" shows: jakowenko/double-take beta-0.7.0 136b9f1cea7e 13 hours ago 910MB

If I remove the camera part from config.yml, the message is the same. I expected { error: "camera config not found" }. Strange...

Inside the container I checked your changes from the beta branch:

/double-take/api/src/controllers # ls -la
total 36
drwxr-xr-x    2 root     root          4096 Jun 18 09:23 .
drwxr-xr-x    7 root     root          4096 Jun 18 09:23 ..
-rw-r--r--    1 root     root           743 Jun 15 04:18 config.controller.js
-rw-r--r--    1 root     root           841 Jun 15 04:18 fs.controller.js
-rw-r--r--    1 root     root          2565 Jun 15 04:18 match.controller.js
-rw-r--r--    1 root     root          4845 Jun 15 04:18 recognize.controller.js
-rw-r--r--    1 root     root          3041 Jun 15 04:18 storage.controller.js
-rw-r--r--    1 root     root          3254 Jun 15 04:18 train.controller.js

cameras.controller.js is missing here. Maybe your changes are not in the beta image?

tunip commented 3 years ago

Now I manually added the changes from the beta branch to my running container.

Now I get: {"id":"978379c7-7276-47d9-ac76-aae4fe845b4e","duration":0.04,"timestamp":"21.06.2021, 17:10:44","attempts":0,"camera":"eingang","zones":[],"matches":[]}

I am directly in front of the camera but no snapshot is taken. Maybe it is now the query string you mentioned earlier.

jakowenko commented 3 years ago

Now I manually added the changes from the beta branch to my running container.

Now I get: {"id":"978379c7-7276-47d9-ac76-aae4fe845b4e","duration":0.04,"timestamp":"21.06.2021, 17:10:44","attempts":0,"camera":"eingang","zones":[],"matches":[]}

I am directly in front of the camera but no snapshot is taken. Maybe it is now the query string you mentioned earlier.

Glad you at least got a response back. Can you try without the query string param for now? http://username:password@192.168.5.19/cgi-bin/snapshot.cgi.

attempts: 0 shows it wasn't able to validate the image url, most likely due to the query param. I'll work on an update tonight to handle query strings.

jakowenko commented 3 years ago

@tunip I confirmed that query string params do work in the config. Can you verify the URL you are using can be resolved by the container?

tunip commented 3 years ago

Camera is reachable from the double-take container. Found the following in the double-take docker logs: url validation error: Request failed with status code 401

Installed wget inside the container and did a successful test:

/double-take # wget -O /tmp/pic.jpg http://username:*password*@192.168.5.19/cgi-bin/snapshot.cgi
--2021-06-22 12:47:53--  http://username:*password*@192.168.8.17/cgi-bin/snapshot.cgi
Connecting to 192.168.5.19:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Syntax error in Set-Cookie: secure; HttpOnly at position 7.
Authentication selected: Digest realm="Login to bf7b617d97fda84e346a02fcce24ab93", qop="auth", nonce="125234566", opaque="28f8aadbf733ad9c78561fd9a92d62b2d743e456"
Connecting to 192.168.5.19:80... connected.
HTTP request sent, awaiting response... 200 OK
Syntax error in Set-Cookie: secure; HttpOnly at position 7.
Length: 683684 (668K) [image/jpeg]
Saving to: '/tmp/pic.jpg'

/tmp/pic.jpg                                 100%[===========================================================================================>] 667.66K  --.-KB/s    in 0.1s    

2021-06-22 12:47:54 (5.06 MB/s) - '/tmp/pic.jpg' saved [683684/683684]

We see in the first http request a 401: HTTP request sent, awaiting response... 401 Unauthorized

The second http request is authenticated and the snapshot could be downloaded: HTTP request sent, awaiting response... 200 OK

I guess in my case it is a authentication issue at the end.

jakowenko commented 3 years ago

Thanks for the update @tunip, let me know if there's anything I can do to assist.

jakowenko commented 3 years ago

Closing since this is now included in the v0.7.0 release. Feel free to open a new issue if you're having problems or have any suggestions.

Raphael909 commented 3 years ago

Good work man! It is very much appreciated!. Sorry, got into some unavoidable circumstance so could not test this quickly .. but beauty of open source community.. thank you @jakowenko and @tunip :)