Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics [ECCV 2024]

Shuai Yang$^{{*}}$, [ZhiFei Chen]()$^{{*}}$, [Pengguang Chen](), [Xi Fang](), [Yixun Liang](), [Shu Liu](), Yingcong Chen$^{**}$

HKUST(GZ), HKUST, SmartMore Corp.

${*}$: Equal contribution. **: Corresponding author.

🎏 Introduction

• We introduce the Defect Spectrum, a comprehensive benchmark that offers precise, semantic-abundant, and large-scale annotations for a wide range of industrial defects.
• We introduce the Defect-Gen, a generator designed to create high-quality and diverse defective images, even when working with limited defective data.

💡 Dataset Reannotation

Industrial datasets often lack detailed defect annotations, providing only binary masks or misclassifications. We introduce the Defect Spectrum, a comprehensive dataset with refined, large-scale annotations for various industrial defects. Using four industrial benchmarks, Defect Spectrum enhances annotation accuracy, capturing subtle and previously missed defects. Our dataset includes rich semantic annotations, identifying multiple defect types per image, and offers descriptive captions for each sample, facilitating future Vision Language Model research.

Example annotation from MVTec AD Dataset

💡 Defect-Gen

Furthermore, we introduce Defect-Gen, a two-stage diffusion-based generator designed to create high-quality and diverse defective images, even when working with limited defective data. The synthetic images generated by Defect-Gen significantly enhance the performance of defect segmentation models, achieving an improvement in mIoU scores up to 9.85 on Defect-Spectrum subsets.

Two-staged defect generation pipeline

This generative model excels in producing diverse and high-quality images, even when trained on limited data.

Qualitative Comparison between Defect-Gen and other generation methods

🛠️ Installation

1. Create an environment with python==3.8.0 conda create -n diff python==3.8.0.
2. Activate it conda activate diff.
3. Install basic requirements pip install -r requirements.txt.

🚀 Getting Started

Train your own Defect-Gens

1. Specify the number of defect types in train_[large/small].sh that corresponds to your own needs. e.g. If the "Capsule" object has 7 defective classes, set the --num_defect to 7.
2. Prepare your config yaml file for both large and small models. The -input channel and -output channel should be a total of the number of defect types, RGB channels, and background channels(if needed). e.g. For an object that has 7 defective classes, the number of input/output channels should be set to 10. (excluding background)
3. Run it with sh train_[large/small].sh

🚀 Inference

1. All the checkpoints will be save to /[working_dir]/checkpoint.
2. Update your checkpoints in inference.sh.
3. Specify your switching point for the large and small model in --step_inference.
4. Specify your defective types in --num_defect.
5. Run inference with sh inference.sh.

📍 Citation

If you find this project useful in your research, please consider citing:

@misc{yang2023defect,
      title={Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics}, 
      author={Shuai Yang and Zhifei Chen and Pengguang Chen and Xi Fang and Shu Liu and Yingcong Chen},
      year={2023},
      eprint={2310.17316},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

• This work is built upon the Guided-Diffusion and SinDiffusion.
• The datasets we adopted come from Apple-Vision, MVTec-AD, DAGM-2007 and Cotton.
• We would like to extend our greatest thanks to those who helped at SmartMore Corp., regardless of whether their contributions are recognized or remain behind the scenes.