fluque1995 / tfm-anomaly-detection

Master's thesis in Data Sciences. Comparison and proposal of enhancement for "Real-world anomaly detection in surveillance videos"
GNU General Public License v3.0
24 stars 7 forks source link

Deep Learning for Crowd Behavior Analysis in Videosurveillance

Master's thesis in Data Sciences: Study on the use of Deep Learning for Crowd Behavior Analysis from videosurveillance sources.

Documents

Main report (in spanish) can be donwloaded from here.

Slides used in the public defense (also in spanish) can be downloaded from here.

Theoretical study

Theoretical study consists of a proposal of taxonomy for crowd behavior analysis, published on Information Fusion with the title Revisiting crowd behavior analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects, which can be found in https://www.sciencedirect.com/science/article/pii/S1566253520303201.

Experimental analysis

In the experimental analysis, we have studied the usage of spatio-temporal features extracted by deep learning models for crowd anomaly detection. Specifically, we have proposed an enhancement over the model in Real-world Anomaly Detection in Surveillance Videos (https://arxiv.org/abs/1801.04264). Instead of using 3D convolutional features, we propose a model which employs convolutional analysis for frames together with a recurrent network (specifically, an LSTM model) to learn the temporal structure of the convolutional features.

Experiments show that our spatio-temporal extractor outperforms the original proposal by a decent margin, even when is pretrained on a smaller dataset for video classification.

Baseline implementations

This implementation, specially the original model replica (which can be found in original_model folder) strongly relies in these previous works:

The original model has been adapted in order to be self-contained in this repo and fully executable in Python. Original proposals rely on external resources and MATLAB for some of the executions, while our implementation is completely designed in Python, which ease the execution.

Software requirements

The project is completely written in Python 3, using the following libraries:

A requirements file is provided for pip installation. In order to install dependencies, navigate to the project root folder and execute:

pip install -r requirements.txt

Data folders structure and datasets

In order to properly execute the models, some folders must be created in advance. Executing the script create_data_folders.sh at root project level will create the required folders with their default names. Also, datasets must be downloaded. In particular:

WARNING: Datasets are heavy, and models are resource-consuming. We strongly recommend using dedicated GPUs and computing nodes to replicate the experiments, since usual PCs are not capable of handling such volumes of data.

Pretrained models

We provide several pretrained models used in our experiments:

Code structure

Developed code is placed in two main folders, together with some scripts to calculate results:

There is more information on how to reproduce the experiments in the README files inside each folder.