Master's thesis in Data Sciences: Study on the use of Deep Learning for Crowd Behavior Analysis from videosurveillance sources.
Main report (in spanish) can be donwloaded from here.
Slides used in the public defense (also in spanish) can be downloaded from here.
Theoretical study consists of a proposal of taxonomy for crowd behavior analysis, published on Information Fusion with the title Revisiting crowd behavior analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects, which can be found in https://www.sciencedirect.com/science/article/pii/S1566253520303201.
In the experimental analysis, we have studied the usage of spatio-temporal features extracted by deep learning models for crowd anomaly detection. Specifically, we have proposed an enhancement over the model in Real-world Anomaly Detection in Surveillance Videos (https://arxiv.org/abs/1801.04264). Instead of using 3D convolutional features, we propose a model which employs convolutional analysis for frames together with a recurrent network (specifically, an LSTM model) to learn the temporal structure of the convolutional features.
Experiments show that our spatio-temporal extractor outperforms the original proposal by a decent margin, even when is pretrained on a smaller dataset for video classification.
This implementation, specially the original model replica (which
can be found in original_model folder) strongly relies in
these previous works:
The original model has been adapted in order to be self-contained in this repo and fully executable in Python. Original proposals rely on external resources and MATLAB for some of the executions, while our implementation is completely designed in Python, which ease the execution.
The project is completely written in Python 3, using the following libraries:
A requirements file is provided for pip installation. In order to
install dependencies, navigate to the project root folder and execute:
pip install -r requirements.txtIn order to properly execute the models, some folders must be created
in advance. Executing the script create_data_folders.sh at root
project level will create the required folders with their default
names. Also, datasets must be downloaded. In particular:
dataset/test.dataset/train/normaldataset/train/abnormalWARNING: Datasets are heavy, and models are resource-consuming. We strongly recommend using dedicated GPUs and computing nodes to replicate the experiments, since usual PCs are not capable of handling such volumes of data.
We provide several pretrained models used in our experiments:
weights_L1L2.mat) and the replica trained by us
(weights_own.mat). These models can be downloaded from
here.
The uncompressed folder must be placed in
original_model/trained_models folderproposal/trained_models
folderDeveloped code is placed in two main folders, together with some scripts to calculate results:
calculate_video_level_scores.py: It calculates the percentage of
normal and abnormal videos in which an alarm has been triggered. For
normal videos, a lesser percentage means lesser false alarms, and
thus a better model. For abnormal videos, a greater percentage means
better capability of detection anomalies.overlay_curves.py: This script computes the ROC and PR curves
given the predictions of both models, and represents them in two
different graphs (one for ROCs and one for PRs).original_model folder: The code in this folder is prepared to
replicate the original experiments, from feature extraction with C3D
to training and evaluation of the anomaly classifier.proposal folder: The code in this folder is prepared to replicate
our experiments. There are scripts to train the feature extractor
over UCF-101, extract features from UCF-Crime dataset using the
pretrained extractor, train and evaluate the anomaly classifier.There is more information on how to reproduce the experiments in the README files inside each folder.