Model Parsing (IEEE TPAMI)

Official Pytorch implementation of our T-PAMI paper "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images".

The paper and supplementary can be found at Arxiv

Authors: Vishal Asnani, Xi Yin, Tal Hassner & Xiaoming Liu.

1. :fire: NEWS :fire:

• [2023/07/29] Our Paper is accepted to Transactions on Pattern Analysis and Machine Intelligence!!

• [2023/07/29] The version 3 of the code is released! The old codebase is deleted. Main branch represents version 3.

• [2022/06/30] The version 2 of the code is released!

• [2021/06/15] In collaboration with Meta AI, our model parsing work is widely reported in CNBC, CNET, Engadget, Fortune, The Mac Observer, MSU Today, New Scientist, SiliconAngle, VentureBeat, The Verge, and Wall Street Journal.

• [2021/05/06] Our codebase for model parsing is released is released!

2. Overview

3. Training/testing

Prerequisites

• PyTorch 1.5.0
• Numpy 1.14.2
• Scikit-learn 0.22.2

Datasets

We collect a large scale dataset comprising of fake images images genearted by 116 generative models. Please visit link for more details. For reverse enginnering:

• Download the model parsing dataset and the ground truth files by following the instructions from here.
• For leave out experiment, put the training data in train folder and leave out models data in test folder
• For testing on custom images, put the data in test folder.

For deepfake detection:

• Download the CelebA/LSUN dataset

For image_attribution:

• Generate 110,000 images for four different GAN models as specified in https://github.com/ningyu1991/GANFingerprints/
• For real images, use 110,000 of CelebA dataset.
• For training: we used 100,000 images and remaining 10,000 for testing.

Training

• Provide the train and test path in respective codes as sepecified below.
• Provide the model path to resume training
• Run the code

For reverse engineering, run:

python reverse_eng.py

For deepfake detection, run:

python deepfake_detection.py

For image attribution, run:

python image_attribution.py

Testing using pre-trained models

• Provide the test path in respective codes as sepecified below
• Download the pre-trained models from https://drive.google.com/drive/folders/1bzh9Pvr7L-NyQ2Mk-TBSlSq4TkMn2anB?usp=sharing
• Provide the model path in the code
• Run the code

For reverse engineering, run:

python reverse_eng_test.py

For deepfake detection, run:

python deepfake_detection_test.py

For image attribution, run:

python image_attribution_test.py

If you would like to use our work, please cite:

@misc{asnani2023reverse,
      title={Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images}, 
      author={Vishal Asnani and Xi Yin and Tal Hassner and Xiaoming Liu},
      journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year={2023}
}