OmniAnomaly is a stochastic recurrent neural network model which glues Gated Recurrent Unit (GRU) and Variational auto-encoder (VAE), its core idea is to learn the normal patterns of multivariate time series and uses the reconstruction probability to do anomaly judgment.
git clone https://github.com/smallcowbaby/OmniAnomaly && cd OmniAnomaly
SMD (Server Machine Dataset) is in folder ServerMachineDataset
.
You can get the public datasets (SMAP and MSL) using:
wget https://s3-us-west-2.amazonaws.com/telemanom/data.zip && unzip data.zip && rm data.zip
cd data && wget https://raw.githubusercontent.com/khundman/telemanom/master/labeled_anomalies.csv
(virtualenv is recommended)
pip install -r requirements.txt
python data_preprocess.py <dataset>
where <dataset>
is one of SMAP
, MSL
or SMD
.
python main.py
If you want to change the default configuration, you can edit ExpConfig
in main.py
or overwrite the config in main.py
using command line args. For example:
python main.py --dataset='MSL' --max_epoch=20
Dataset name | Number of entities | Number of dimensions | Training set size | Testing set size | Anomaly ratio(%) |
---|---|---|---|---|---|
SMAP | 55 | 25 | 135183 | 427617 | 13.13 |
MSL | 27 | 55 | 58317 | 73729 | 10.72 |
SMD | 28 | 38 | 708405 | 708420 | 4.16 |
SMAP (Soil Moisture Active Passive satellite) and MSL (Mars Science Laboratory rover) are two public datasets from NASA.
For more details, see: https://github.com/khundman/telemanom
SMD (Server Machine Dataset) is a new 5-week-long dataset. We collected it from a large Internet company. This dataset contains 3 groups of entities. Each of them is named by machine-<group_index>-<index>
.
SMD is made up by data from 28 different machines, and the 28 subsets should be trained and tested separately. For each of these subsets, we divide it into two parts of equal length for training and testing. We provide labels for whether a point is an anomaly and the dimensions contribute to every anomaly.
Thus SMD is made up by the following parts:
concatenate
With the default configuration, main.py
follows these steps:
train_score.pkl
and test_score.pkl
.train_score
to find the threshold of anomaly score, and using this threshold to predict on the testing set.The figure below are the training loss of our model on MSL and SMAP, which indicates that our model can converge well on these two datasets.