Wavelet-Packets for Deepfake Image Analysis and Detection

This is the supplementary source code for our paper Wavelet-Packets for Deepfake Image Analysis and Detection, Machine Learning, Special Issue of the ECML PKDD 2022 Journal Track.

The plot above illustrates the fundamental principle. It shows an FFHQ and a style-gan-generated image on the very left. In the center and on the right, packet coefficients and their standard deviation are depicted. We computed mean and standard deviation values using 5k images from each source.

Installation

The latest code can be installed in development mode with:

$ git clone https://github.com/gan-police/frequency-forensics
$ cd frequency-forensics
$ pip install -e .

Assets

GAN Architectures

We utilize pre-trained models from the following repositories:

• StyleGAN
• GANFingerprints

For our wavelet-packet computations, we use the :

• PyTorch-Wavelet-Toolbox: ptwt

In the paper, we compare our approach to the DCT-method from:

• GANDCTAnalysis

Datasets

We utilize three datasets that commonly appeared in previous work:

• FFHQ
• CelebA
• LSUN bedroom

Reproduction

The following section of the README serves as a guide to reproducing our paper. Data for the 128 pixel FFHQ-Stylegan pair is available via google-drive [4.2GB] .

Preparation

We work with images of the size 128x128 pixels. Hence, the raw images gave to be cropped and/or resized to this size. To do this, run freqdect.crop_celeba or freqdect.crop_lsun, depending on the dataset. This will create a new folder with the transformed images. The FFHQ dataset is already distributed in the required image size.

Use the pretrained GAN-models to generate images. In case of StyleGAN, there is only a pre-trained model generating images of size 1024x1024, so one has to resize the GAN-generated images to size 128x128 pixels, e.g. by inserting

PIL.Image.fromarray(images[0], 'RGB').resize((128, 128)).save(png_filename)

into the ffhq-stylegan .

Store all images (cropped original and GAN-generated) in a separate subdirectories of a directory, i.e. the directory structure should look like this

source_data
 ├── A_original
 ├── B_CramerGAN
 ├── C_MMDGAN
 ├── D_ProGAN
 └── E_SNGAN

For the FFHQ case, we have only two subdirectories: ffhq_stylegan/A_ffhq and ffhq_stylegan/B_stylegan. The prefixes of the folders are important, since the directories get the labels in lexicographic order of their prefix, i.e. directory A_... gets label 0, B_... label 1, etc.

Now, to prepare the data sets run freqdect.prepare_dataset . It reads in the data set, splits them into a training, validation and test set, applies the specified transformation (to wavelet packets, log-scaled wavelet packets or just the raw image data) and stores the result as numpy arrays.

Afterwards run e.g.:

$ python -m freqdect.prepare_dataset ./data/source_data/ --log-packets 
$ python -m freqdect.prepare_dataset ./data/source_data/

The dataset preparation script accepts additional arguments. For example, it is possible to change the sizes of the train, test or validation sets. For a list of all optional arguments, open the help page via the -h argument.

Training the Classifier

Now you should be able to train a classifier using for example:

$ python -m freqdect.train_classifier \
    --data-prefix ./data/source_data_log_packets_haar_reflect_3 \
    --calc-normalization \
    --features packets

This trains a regression classifier using default hyperparameters. The training, validation and test accuracy and loss values are stored in a file placed in a log folder. The state dict of the trained model is stored there as well. For a list of all optional arguments, open the help page via the -h argument.

Evaluating the Classifier

Plotting the Metrics

To plot the accuracy results, run:

$ python -m freqdect.plot_accuracy_results {shared, lsun, celeba} {regression, CNN} ...

For a list of all optional arguments, open the help page via the -h argument.

Calculating the confusion matrix

To calculate the confusion matrix, run freqdect.confusion_matrix. For a list of all arguments, open the help page via the -h argument.

⚖️ Licensing

This project is licensed under the GNU GPLv3 license

Acknowledgements

📖 Citation

If you find this work useful please consider citing:

@article{wolter2022waveletpacket,
      title   = {Wavelet-Packets for Deepfake Image Analysis and Detection}, 
      author  = {Moritz Wolter and Felix Blanke and Raoul Heese and Jochen Garcke},
      journal = {Machine Learning},
      year    = {2022},
      volume  = {Special Issue of the ECML PKDD 2022 Journal Track},
      pages   = {1-33},
      month   = {August},
      url     = {https://rdcu.be/cUIRt},
      issn    = {0885-6125}, 
      doi     = {https://doi.org/10.1007/s10994-022-06225-5}
}

🙏 Support

This project has been supported by the following organizations (in alphabetical order):

• Fraunhofer Institute for Algorithms and Scientific Computing (SCAI)
• Fraunhofer Cluster of Excellence Cognitive Internet Technologies (CCIT)

🍪 Cookiecutter

This package was created with @audreyfeldroy's cookiecutter package using @cthoyt's cookiecutter-snekpack template.