search

awsaf49 / artifact

[ICIP 2023] ArtiFact: A Large-Scale Dataset with Artificial (Fake) and Factual (Real) Images for Generalizable and Robust Synthetic Image Detection
https://arxiv.org/abs/2302.11970
59 stars 5 forks source link
cnn comuter-vision dataset deep-learning diffusion gan real-vs-fake transformer

readme

ArtiFact: A Large-Scale Dataset with Artificial and Factual Images for Generalizable and Robust Synthetic Image Detection [ICIP 2023]

Paper:

Abstract: Synthetic image generation has opened up new opportunities but has also created threats in regard to privacy, authenticity, and security. Detecting fake images is of paramount importance to prevent illegal activities, and previous research has shown that generative models leave unique patterns in their synthetic images that can be exploited to detect them. However, the fundamental problem of generalization remains, as even state-of-the-art detectors encounter difficulty when facing generators never seen during training. To assess the generalizability and robustness of synthetic image detectors in the face of real-world impairments, this paper presents a large-scale dataset named ArtiFact, comprising diverse generators, object categories, and real-world challenges. Moreover, the proposed multi-class classification scheme, combined with a filter stride reduction strategy addresses social platform impairments and effectively detects synthetic images from both seen and unseen generators. The proposed solution significantly outperforms other top teams by 8.34% on Test 1, 1.26% on Test 2, and 15.08% on Test 3 in the IEEE VIP Cup challenge at ICIP 2022, as measured by the accuracy metric.

Presentation: YouTube

Visual Summary:

Update

Result on IEEE VIP Cup at ICIP 2022

Accuracy (%) of Top3 Teams on Leaderboard,

Team Names Test 1 Test 2 Test 3
Sherlock 87.70 77.52 73.45
FAU Erlangen-Nürnberg 87.14 81.74 75.52
Megatron (Ours) 96.04 83.00 90.60

Note: A small portion of the proposed ArtiFact dataset, totaling 222K images of 71K real images and 151K fake images from only 13 generators is used in the IEEE VIP Cup. Here all the Test data is kept confidential from all participating teams. Additionally, the generators used for the Test 1 data are known to all teams, whereas the generators for Test 2 and Test 3 are kept undisclosed.

Dataset Description

Data Distribution

Download Dataset

The dataset is hosted on Kaggle. The dataset can be downloaded i) directly from the browser using the link below or ii) can be downloaded using kaggle-api.

i) Directly from Browser

Link: ArtiFact Dataset

ii) Kaggle API

!kaggle datasets download -d awsaf49/artifact-dataset

How to Use

The dataset is organized into folders, each of which corresponds to a specific generator of synthetic images or source of real images. Each folder contains a metadata.csv file, which provides information about the images in the folder. It contains following columns,

Data Generation

!python data/transform.py <input directory> <output directory> <seed>

Citation

@INPROCEEDINGS{artifact,
  author={Rahman, Md Awsafur and Paul, Bishmoy and Sarker, Najibul Haque and Hakim, Zaber Ibn Abdul and Fattah, Shaikh Anowarul},
  booktitle={2023 IEEE International Conference on Image Processing (ICIP)}, 
  title={Artifact: A Large-Scale Dataset With Artificial And Factual Images For Generalizable And Robust Synthetic Image Detection}, 
  year={2023},
  volume={},
  number={},
  pages={2200-2204},
  doi={10.1109/ICIP49359.2023.10222083}}

License

ArtiFact dataset takes leverage of data from multiple methods thus different parts of the dataset come with different licenses. All the methods and their associated licenses are mentioned in the table,

Data License | Method | License | |:------------------------|:---------------------------------------------------------------------------------------| | ImageNet | Non Commercial | | COCO | Creative Commons Attribution 4.0 License | | LSUN | Unknown | | AFHQ | Creative Commons Attribution-NonCommercial 4.0 International Public | | FFHQ | Creative Commons BY-NC-SA 4.0 license | | Metfaces | Creative Commons BY-NC 2.0 | | CelebAHQ | Creative Commons Attribution-NonCommercial 4.0 International Public | | Landscape | MIT license | | Glide | MIT license | | StyleGAN2 | Nvidia Source Code License | | StyleGAN3 | Nvidia Source Code License | | Generative Inpainting | Creative Commons Public Licenses | | Taming Transformer | MIT License | | MAT | Creative Commons Public Licenses | | LaMa | Apache-2.0 License | | Stable Diffusion | Apache-2.0 License | | VQ Diffusion | MIT License | | Palette | MIT License | | StyleGAN1 | Creative Commons Public Licenses | | Latent Diffusion | MIT License | | CIPS | MIT License | | StarGAN | MIT License | | BigGAN | MIT License | | GANformer | MIT License | | ProjectedGAN | MIT License | | SFHQ | MIT License | | FaceSynthetics | Research Use of Data Agreement v1.0 | | Denoising Diffusion GAN | NVIDIA License | | DDPM | Unknown | | DiffusionGAN | MIT License | | GauGAN | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License | | ProGAN | Attribution-NonCommercial 4.0 International | | CycleGAN | BSD |

Acknowledgment