Real3D-AD: A Dataset of Point Cloud Anomaly Detection

Jiaqi Liu, Guoyang Xie, Ruitao Chen*, Xinpeng Li, Jinbao Wang†, Yong Liu, Chengjie Wang, and Feng Zheng†

(* Equal contribution; † Corresponding authors)

Our paper has been accepted by NeurIPS 2023 Datasets & Benchmarks Track. [Paper]

Overview

This project aims to construct a new dataset of high-resolution 3D point clouds for anomaly detection tasks in real-world scenes.

Real3D-AD can be used for training and testing 3D anonmaly detection algorithms.

Note that different from RGB + Depth patterns, we only provide 3D point clouds for users.

Real3D-AD

Summary

• overview of all classes in Real3D-AD

Real3D-AD comprises a total of 1,254 samples that are distributed across 12 distinct categories. These categories include Airplane, Car, Candybar, Chicken, Diamond, Duck, Fish, Gemstone, Seahorse, Shell, Starfish, and Toffees.

Download

• To download the Real3D-AD dataset (Dataset for training and evaluation, pcd format), click real3d-ad-pcd.zip(google drive) or real3d-ad-pcd.zip(baidu disk: vrmi)
• To download the Real3D-AD dataset (Source data from camera, ply format), click real3d-ad-ply.zip(google drive) or real3d-ad-ply.zip(baidu disk：vvz1)

Data preparation

• Download real3d-ad-pcd.zip and extract into ./data/

  data
  ├── airplane
  ├── train
      ├── 1_prototype.pcd
      ├── 2_prototype.pcd
      ...
  ├── test
      ├── 1_bulge.pcd
      ├── 2_sink.pcd
      ...
  ├── gt
      ├── 1_bulge.txt
      ├── 2_sink.txt
      ... 
  ├── car
  ...

Checkpoint preparation

• Download checkpoints and move them into ./checkpoints/

Dataset Statistic

• brief describe our dataset in Table

(Δ: Mean proportion of abnormal point clouds in Test set)

Data Collection

• description of instruments

The PMAX-S130 optical system comprises a pair of lenses with low distortion properties, a high luminance LED, and a blue-ray filter. The blue light scanner is equipped with a lens filter that selectively allows only the blue light of a specific wavelength to pass through. The filter effectively screens the majority of blue light due to its relatively low concentration in both natural and artificial lighting. Nevertheless, using blue light-emitting light sources could pose a unique obstacle in this context. The image sensor can collect light using the lens aperture. Hence, the influence exerted by ambient light is vastly reduced.

• how to capture point clouds and complete one prototype

Initially, the stationary object undergoes scanning while the turntable completes a full revolution of 360°, enabling the scanner to capture images of the various facets of the object. Subsequently, the object undergoes reversal, and the process of rotation and scanning is reiterated. Following the manual calibration of the front and back scanning outcomes, the algorithm performs a precise calibration of the stitching process. If there are any gaps in the stitching outcome, the scan stitching process is reiterated until the point cloud is rendered.

• anomalies

The anomalies pertaining to point clouds can be classified into two categories: incompleteness and redundancy. In the dataset, we named them bulge and sink. Besides, more samples are made by copying and cutting edges.

Annotation

• how to annotate

The collected point clouds are annotated using CloudCompare software CloudCompare is a 3D point cloud (grid) editing and processing software. Originally, it was designed to directly compare dense three-dimensional point clouds. It relies on a specific octree structure and provides excellent performance for tasks such as point cloud comparison. The anotation process of point cloud is shown in the figure below.

Benchmark

• beseline methods

We take BTF and M3DM as basic baseline methods, and improve baseline using PatchCore.

• metrics

We choose AUROC and AUPU as metric for object level and point level anomaly detection.

• benchmark results

Other methods on Real3D-AD

Complementary Pseudo Multimodal Feature for Point Cloud Anomaly Detection [paper 2023][code] Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network [paper 2023][code]

• PointCore: Efficient Unsupervised Point Cloud Anomaly Detector Using Local-Global Features [paper 2024]

Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network [paper][code]

The 3D anomaly detection performance when using 4 prototypes for training.

How to reproduce our benchmark

• environment preparation

We implement benchmark under CUDA 11.3 Our environment can be reproduced by the following command:

conda env create -f real3dad.yaml
# Note that point2_ops_lib may need to be installed by the following command：
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"

• how to train and evaluation

Note: Although we have fixed the random seed, slight variations in the results may still occur due to other random factors. We will try to address this issue in future work.

sh start.sh

The result will output four different metrics (Object/Point AUROC/AUPR).

• how to visualize abnormal map

In util.visualization.py, we provide the function ''vis_pointcloud_gt'' to visualize ground truth with our gt files. Also, with a saved anomaly map xx.npy(n values between 0 and 1) and the corresponding pcd file(n xyz points), you can use "vis_pointcloud_anomalymap(point_cloud, anomaly_map)" to visualize anomaly regions.

Acknowledgments.

This work is supported by the National Key R&D Program of China (Grant NO. 2022YFF1202903) and the National Natural Science Foundation of China (Grant NO. 62122035, 62206122).

Our benchmark is built on BTF and M3DM and PatchCore, thanks their extraordinary works!

Thanks to all the people who worked hard to capture the data, especially Xinyu Tang for his efforts.

License

The dataset is released under the CC BY 4.0 license.

BibTex Citation

If you find this paper and repository useful, please cite our paper☺️.

@inproceedings{liu2023real3d,
  title={Real3D-AD: A Dataset of Point Cloud Anomaly Detection},
  author={Liu, Jiaqi and Xie, Guoyang and Li, Xinpeng and Wang, Jinbao and Liu, Yong and Wang, Chengjie and Zheng, Feng and others},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year={2023}
}