Smart Human Activity Detection Using YOLO
This is a project to perform fall detection, vehicle crash detection and social distancing detection from CCTV cameras in real-time.
YOLO stands for You Only Look Once. It is used for object detection To perform object detection on an image it looks at an image only once in a very clever way unlike R-CNN which takes several instances of the same image to perform detection.
YOLO divides an image into a grid and several bounding boxes are formed. Then a confidence score is taken for each boundary box to see whether an bounding box contains any object within it. The confidence score is high if the object inside the box matches the pre-trained YOLO dataset ( COCO Dataset ). The higher the confidence score, the higher the probability that a bounding box contains an object. Now several bounding boxes will intersect with each other. More the bounding boxes intersect, more is the probability that there is an object inside that box. Now we only keep those bounding boxes whose confidence score is more than threshold value lets say 30%. Now we match these bounding boxes with already known features of an object like person, car and classify them.
The good thing about YOLO is that all the predictions in the boxes are made at the same time i.e. the neural networks just ran only once. And that is why YOLO is powerful and fast.
You can follow the two part YouTube videos of Augmented Startups
After the above installation work is done and darknet libraries are working, place the python files inside [YOLO\darknet\build\darknet\x64]() folder.
There are four ways to perform detection on videos:
See the code of the program and uncomment the line from which you want to take the video to perform detection.
You can take the video from Sample dataset to perform detection.
Simple YOLO program for object detection.
python Object_Detection.py
We take the input video from a source and divide the video into several frames. Now these frames are converted into black and white. On each frame a person is detected using YOLO.
Now we write the code to draw rectangles on the detected persons. Whenever the height of the rectangle is greater than width of the rectangle Fall is not detected and when width is greater than height Fall is detected.
And this is how we classify the images into a fall and not fall and an alert is generated if a fall is detected.
All of the above process happens for a single frame. Now all of this is set in a loop for each frame of the video and Fall is detected.
If the fall is detected in 20 frames of a video simultaneously then an alert is generated by sending an email to the required person. That email also contains the photo of the fall that was detected.
image_email_fall.py
file]2 step verification
is OFF
and less secure app access
is ON
for your email account]python Fall_Detection.py
We take the input video from a source and divide the video into several frames. Now these frames are converted into black and white. On each frame a person is detected using YOLO.
Now we write the code to draw rectangles on the detected persons. We check the distances between each detected person on the frame from each other. If the distance between the two persons is less than a particular value then we colour the box red and draw a line between these boxes and add the no. of social distancing violations in a variable and display it.
All of the above process happens for a single frame. Now all of this is set in a loop for each frame of the video and People at Risks are detected.
python Social_Distance.py
We take the input video from a source and divide the video into several frames. Now these frames are converted into black and white. On each frame a car is detected using YOLO.
Now we write the code to draw rectangles on the detected cars. We check the distances between each detected car on the frame from each other. If the distance between the two cars is less than a particular value and the rectangle boxes of any two cars intersect each other then we colour the box red display the message that Crash has been detected.
All of the above process happens for a single frame. Now all of this is set in a loop for each frame of the video and Vehicle crash is detected.
If the car crash is detected in 20 frames of a video simultaneously then an alert is generated by sending an email to the required person. That email also contains the photo of the car crash that was detected.
image_email_car.py
file]2 step verification
is OFF
and less secure app access
is ON
for your email account]python Vehicle_Crash.py
We have deployed our project using flask. When we run the below script, our website is hosted onto a local server and we can use that website to perform detections on videos.
python app.py
After installing darknet keep the github files according to this file structure
darknet
|
|
|───build
| |
| |
| |───darknet
| | |
| | |
| | |───x64
| | | |
| | | |
| | | |───app.py
| | | |───Fall_Detection.py
| | | |───Object_Detection.py
| | | |───Social_Distance.py
| | | |───Vehicle_Crash.py
| | | |───image_email.py
| | | |
| | | |───templates
| | | | |
| | | | |
| | | | |───ContactUs.html
| | | | |───FallDetection.html
| | | | |───ObjectDetection.html
| | | | |───Shady.html
| | | | |───SocialDistancingDetection.html
| | | | |───VehicleCrashDetection.html
| | | | |───Video.html
| | | |
| | | |───static
| | | | |
| | | | |
| | | | |───assets
| | | | |
| | | | |
| | | | |───...files...
| | | | |───fonts
| | | | |
| | | | |
| | | | |───...files...
| | | |───...files...
| | |───...files...
| |───...files...
|───...files...