PatchCore Anomaly Detection

This repository provides an unofficial PyTorch implementation of the PatchCore anomaly detection model [1] and several additional experiments.

PatchCore is an anomaly detection algorithm that has the following features:

• uses a memory-bank of nominal features extracted from a pre-trained backbone network (like SPADE and PaDiM), where the memory back is coreset-subsampled to ensure low inference cost at higher performance,
• uses an approximated nearest neighbor search for evaluating pixel-wise anomaly score on inference,
• shows state-of-the-art performance on the MVTec AD dataset (as of Jun 2021).

Usage

Installation

The author recommends using Docker for keeping your environment clean. For example, you can create a new Docker container and enter into it by the following command in the root directory of this repository:

docker run --rm -it -v `pwd`:/workspace -w /workspace --name patchcore tiskw/patchcore:cpu-2022-03-01

If you need GPU support, please use the Docker image with CUDA libraries:

docker run --rm -it -v `pwd`:/workspace -w /workspace --name patchcore tiskw/patchcore:gpu-2022-03-01

See this document for more details.

Dataset

You need to get the MVTec AD dataset [2] if you will reproduce our experiments. If you don't have a plan to use the dataset, you can skip this subsection. You can download the MVTec AD dataset from the official website (or direct link to the data file) and put it under data/mvtec_ad/ directory.

At first, please move to the data/mvtec_ad/ directory.

cd data/mvtec_ad/

Then, run the following command to download the MVTec AD dataset:

wget "https://www.mydrive.ch/shares/38536/3830184030e49fe74747669442f0f282/download/420938113-1629952094/mvtec_anomaly_detection.tar.xz"

Finally, extract the downloaded data:

tar fJx mvtec_anomaly_detection.tar.xz

See this document for more details.

Train and predict on your dataset

You can train and run predictions on your dataset using main.py. In the following, we assume:

• your training images (good images) are stored under data_train/ directory,
• your test images (good or failure images) are stored under data_test/ directory,
• the training result will be stored at ./index.faiss,
• the test results will be dumped under output_test/ directory.

You can train your model by the following command:

python3 main.py train -i data_train -o ./index.faiss

Then, you can predict anomaly score for test images by the following command:

python3 main.py predict -i data_test -o output_test

On the output directory output_test/, two types of files will be dumped:

• .jpg: anomaly heatmap overlayed on the input image,
• .npy: matrix of anomaly heatmap with shape (height, width).

Replicate the experiments

If you want to replicate the experiments, run the following command at the root directory of this repository:

python3 main_mvtecad.py runall
python3 main_mvtecad.py summarize

The python3 main_mvtecad.py runall command will take a quite long time, therefore it is a good idea to wrap the above command by nohup.

Anomalous area visualization

If you want the visualizatino of the anomalous area for each sample like the following figure, you can try --contour option in the prediction step. The following is the details of steps to generate the anomalous area visualization.

At first, the training data is located under data_train/ directory, and we assume that you completed the training of the model:

# Train the PatchCore model.
python3 main.py train -i data_train -o ./index.faiss

Next, we need to compute the threshold for determining the anomalous area for each samples. You can compute the threshold by the following command:

# Compute threshold.
python3 main.py thresh -i data_train

Finally, you can get the anomalouse area visualization by the following command where we assume that the test data is located under data_test/ directory and THRESH is the threshold value computed in the previous step:

# Visualize contour map using the threshold value obtained by the above.
python3 main.py predict --contour THRESH -i data_test -o output_test

Experiments

Experiment 1: Comparison with the original paper on MVTec AD dataset

The following figures are summaries of the comparison of the anomaly detection scores on MVTec AD dataset [2] with the original paper of the PatchCore [1]. The performance of our implementation is quite close to the paper's score, therefore our implementation may have no serious issue.

See this document for more details.

Experiment 2: Comparison of backbone networks

We compared the image/pixel-level scores on the MVTec AD dataset with different backbone networks. Some networks show a better speed/performance tradeoff than Wide ResNet50 x2 which is used as a default backbone network in the original paper.

See this document for more details.

Experiment 3: Comparison of pre-trainings

We compared several different pre-trained ResNet50 as a backbone of PatchCore. We hypothesize that a well-trained neural network achieves higher performance. We tried the normal ImageNet pre-training ResNet50, DeepLabV3 resNet50 pre-trained with COCO, and ResNet50-SSL/SWSL model that are pre-trained on ImageNet [3] with semi-supervise or un-supervised manner. The result is quite interesting, however, basically, we can say that the normal ImageNet pre-trained model is enough good for PatchCore purposes.

See this document for more details.

Notes

• This implementation refers to another PatchCore implementation [6] which is released under Apache 2.0 license. The author has learned a lot from the implementation.

References

[1] K. Roth, L. Pemula, J. Zepeda, B. Scholkopf, T. Brox, and P. Gehler, "Towards Total Recall in Industrial Anomaly Detection", arXiv, 2021. PDF

[2] P. Bergmann M. Fauser D. Sattlegger, and C. Steger, "MVTec AD - A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection", CVPR, 2019. PDF

[3] I. Yalniz, H. Jegou, K. Chen, M. Paluri and D. Mahajan, "Billion-scale semi-supervised learning for image classification", arXiv, 2019. PDF

[4] E. Fix and J. Hodges, ”Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties”, USAF School of Aviation Medicine, Randolph Field, Texas, 1951.

[5] B. Scholkopf, R. Williamson, A. Smola, J. Shawe-Taylor and J. Platt, "Support Vector Method for Novelty Detection", NIPS, 1999. PDF

[6] hcw-00/PatchCore_anomaly_detection, GitHub.