Home Surveillance

[cloudhaus]

I need some help architecting a solution.

I am currently using Blue Iris with 10 cameras around my home. It is working great but I am not satisfied with the motion detection. I cannot find a mix of settings in Blue Iris that I am happy with to detect motion. I do not blame Blue Iris for this. For example on a windy / sunny day I get many false alerts. Blue Iris is very reliable for recording video 24/7 and I would like to keep it.

I would like to use some kind of image classification in addition to Blue Iris. It could be a completely separate system and I would turn off alerts in Blue Iris. I am using Homeseer for home automation.

Goals

• Know when a person is approaching/at the front door.
• Know when a person or car/truck comes onto the driveway.
• Know when a person is around the pool. By "Know" I mean Homeseer knows then I can act on the event.

Challenges

• The cameras (mostly Hikvision) offer http/rtsp direct access to the video.. Accessing the cameras directly instead of through Blue Iris might be quicker.
• Since I have Homeseer but others might have openhab or Home Assistant publish to MQTT might be the best way forward when there is motion. Homeseer can subscribe to MQTT.
• Since there is usually a car parked on my driveway I would need to know when the number of cars goes up. If there are no cars on the driveway currently I need to know when one comes.
• The driveway camera also captures the street. I don't need to know if a car passes on the street. I would need to provide coordinates (a rectangle) for area of interest, ie the driveway. Same for people passing on the street, I only need to know if they approach.
• I would like to have an email of the event with an attached image with boxes around the car or person.

Is this too ambitious? Can anyone help architect a solution? I currently do not have a CUDA GPU but will buy one if this project is possible.

[AlexeyAB]

I used rtsp stream from Hikvision for Darknet Yolo in a such way: darknet.exe detector demo data/coco.data cfg/yolov3.cfg yolov3.weights rtsp://login:pass@192.168.0.228:554/h264/ch01/main/av_stream It will have less latency if you will compile OpenCV with Gstreamer library.

Classifictation usually doesn't count number of objects, so it can't distinguish are there 2 cars or only 1 car. So you can try to use Detection (Yolo) and wrote your own C/C++/Python code to check what objects are there and how many. About "area of interest", you can try to configure it on Hikvision, or you just can check coordinates of objects in your own code - are they in the "area of interest" or not.

[fvlntn]

Of course it is possible.

Just use rtsp stream to get camera stream. (check src/demo.c / http_stream.cpp / image.c to see how Alexey connects to it with opencv) Use a model that detects person and car. Then do your own Python script (based on darknet.py) so you can mathematically count what you want to do (one Python script for each camera on which you want to do detection) (you should check how to connect to rtsp stream from opencv python)

For instance for the car on driveway: Do detection on driveway camera frame. Check if detected cars are inside area of interest (get coordinates of both and check) Count how many cars detected. Whenever it goes up and stays that number for a great amount of successive frames then get alerted. There are Python modules to send image attached to a mail, just save the image with opencv. Then save that number so you can see if another car is parked. And save if it goes down also.

For people in the street, you should try by yourself first and check the bounding box surface for a person on the street and a person approaching your camera. Then just check width*height to get a not-really-good-but-good-enough approximation of the depth of the person.

Only thing is you should launch 3 different darknet instances to do detection for the pool/driveway/frontdoor. Everything else is simple programmation that can be done really fast in Python.