search

neilenns / node-deepstackai-trigger

Detects motion using Deepstack AI and calls registered triggers based on trigger rules.
MIT License
165 stars 28 forks source link

Custom Deepstack Models #416

Closed ghzgod closed 3 years ago

ghzgod commented 3 years ago

Describe the solution you'd like I have created a few custom deepstack models using this tutorial Deepstack Custom Models. I can then ask Deepstack to analyze an image via the custom model link EG: http://10.0.0.X:5000/v1/vision/custom/my_custom_model_name?image which will reply with the confidence of the detected object.

A good example of this would be not only detecting that a car is in the driveway, but who's car is in the driveway? Mine? My friend? etc.

I've been able to save triggered images from node-deepstack-ai by setting the purgeInterval to 0 (thanks for adding that) and then using those images I've collected overtime, to label and train.

So logically...I can see this as follows: 1) My driveway has motion, image stores to aiinput folder 2) Node-deepstack AI sends it to default deepstackUri for analysis. 3) Another trigger config in triggers.json sends that same image to my custom deepstack model link for analysis. 4) From there, I can take actions via MQTT.

Maybe allowing a deepstackUri override in triggers.json? And ignoring things like watchObjects as the only objects that will be detected are

1) A car I know or 2) Not a car I know

aka (A hot dog/not a hot dog 😄 ).

I am sure this is more complex but just an idea. I can't imagine if many people use custom models or not. Thank you in advance.

neilenns commented 3 years ago

Thank you for the suggestion, I have a build for you to try.

Change your docker-compose so instead of using latest it uses issue416. Then in your triggers.json add a customEndpoint property that points to the path for your custom model on each trigger that needs it:

"customEndpoint": "/v1/vision/custom/my_custom_model_name?image"

You still need to specify at least one watchObject but the code will ignore it.

I don't really have any way to test this so let me know if it works. If it does I'll figure out how to make the JSON schema not require watchObjects when customEndpoint is specified.

ghzgod commented 3 years ago

This is working beautifully and I am thankful because my node-red flow was more complex than it needed to be.

My mqtt alert shows the custom trained objects and their predicted values. Perfect! Thank you so much.

If you ever wanted to test this on your own, here is a custom license plate detector model if you have a camera facing a street with car traffic.

neilenns commented 3 years ago

Can you share the JSON you are getting in the mqtt messages? I'd love to see it.

Thanks for the link to the license plate detector. Will see if I can integrate that into the dev stack for testing.

Once I get the schema figured out I'll release the change and let you know when it's is available via latest.

From: ghzgod notifications@github.com Sent: Monday, January 18, 2021 10:38:58 AM To: danecreekphotography/node-deepstackai-trigger node-deepstackai-trigger@noreply.github.com Cc: Neil Enns info@danecreekphotography.com; Assign assign@noreply.github.com Subject: Re: [danecreekphotography/node-deepstackai-trigger] Custom Deepstack Models (#416)

This is working beautifully and I am thankful because my node-red flow was more complex than it needed to be.

My mqtt alert shows the custom trained objects and their predicted values. Perfect! Thank you so much.

If you ever wanted to test this on your own, here is a custom license plate detector modelhttps://github.com/odd86/deepstack_licenceplate_model if you have a camera facing a street with car traffic.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://github.com/danecreekphotography/node-deepstackai-trigger/issues/416#issuecomment-762414312, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACIVHFTSCV6LMOICCZI4UDLS2R54FANCNFSM4WHJZRWQ.

ghzgod commented 3 years ago

Sure, is this what you're looking for?

{ "analysisDurationMs": 4082, "basename": "drivewayhd.20210118_195848070.jpg", "fileName": "/aiinput/drivewayhd.20210118_195848070.jpg", "formattedPredictions": "Toyota (60%), Nissan (67%)", "name": "Driveway Custom", "predictions": [ { "confidence": 0.60056514, "label": "Toyota", "y_min": 26, "x_min": 854, "y_max": 1202, "x_max": 1949 }, { "confidence": 0.6700693, "label": "Nissan", "y_min": 78, "x_min": 16, "y_max": 1236, "x_max": 630 } ], "state": "on" }

neilenns commented 3 years ago

Yep. That's super cool! I think I got the schema figured out, will have a pre-release build for you to try shortly.

neilenns commented 3 years ago

Ok this is now published under the dev image tag. Would you mind updating your config to that and verifying it still works? Also please also try various combinations of watchObjects and customEndpoint: it should complain if you specify both, complain if neither are specified, etc.

ghzgod commented 3 years ago

When you say "it should complain if you specify both" I thought earlier you mentioned that I had to have a value in there (watchObjects). Has that changed since issue416 image?

neilenns commented 3 years ago

Yes, I figured out how to enforce one or the other but not both via the JSON schema.

ghzgod commented 3 years ago

dev tag is working.

As far as triggers.json :

It does not complain if I have both watchObjects and customEndpoint set.

It does not complain if I have only customEndpoint and no watchObjects set.

It does not complain if I have only watchObjects and no customEndpoint set.

It does not complain if I have neither customEndpoint / watchObjects set.

The logs for all 4 of these conditions in triggers.json showed the image being analyzed. The only one that did not return results was when both customEndpoint watchObjects were unset which makes sense.

neilenns commented 3 years ago

Hmm. Ok, thanks. I'll have to do a bit more digging!

neilenns commented 3 years ago

So the reason it doesn't error is because... I never call the validation method 🤦‍♂️ I wonder how long that bug has been there. Sigh.

From: ghzgod notifications@github.com Sent: Monday, January 18, 2021 12:12:28 PM To: danecreekphotography/node-deepstackai-trigger node-deepstackai-trigger@noreply.github.com Cc: Neil Enns info@danecreekphotography.com; State change state_change@noreply.github.com Subject: Re: [danecreekphotography/node-deepstackai-trigger] Custom Deepstack Models (#416)

dev tag is working.

As far as triggers.json :

It does not complain if I have both watchObjects and 'customEndpoint' set.

It does not complain if I have only 'customEndpoint' and no 'watchObjects' set.

It does not complain if I have only 'watchObjects' and no 'customEndpoint' set.

It does not complain if I have neither 'customEndpoint' / 'watchObjects' set.

The logs for all 4 of these conditions in triggers.json showed the image being analyzed. The only one that did not return results was when both customEndpoint watchObjects were unset which makes sense.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHubhttps://github.com/danecreekphotography/node-deepstackai-trigger/issues/416#issuecomment-762450687, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACIVHFTWLN5J3MKBETMRAJLS2SI2ZANCNFSM4WHJZRWQ.

ghzgod commented 3 years ago

Haha the wonderful world of coding. Trust me, I know how you feel.