Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, and Shu Hu
This repository is the official implementation of our paper "Preserving Fairness Generalization in Deepfake Detection", which has been accepted by CVPR 2024.
You can run the following script to configure the necessary environment:
cd Fairness-Generalization
conda create -n FairnessDeepfake python=3.9.0
conda activate FairnessDeepfake
pip install -r requirements.txtWe share the FF++, Celeb-DF, DFD, DFDC with demographic annotations from paper, which be downloaded through this link.
You can also get those re-annotated four datasets with prediction uncertainty scores through our AI-Face-FairnessBench.
Or you can download these datasets from their official website and process them by following the below steps:
Download annotations for these four datasets according to paper and their code, extract the demographics information of all images in each dataset.
Extract, align and crop face using DLib, and save them to /path/to/cropped_images/
Split cropped images in each dataset to train/val/test with a ratio of 60%/20%/20% without identity overlap.
Generate faketrain.csv, realtrain.csv, fakeval.csv, realval.csv according to the following format:
|- faketrain.csv
|_ img_path,label,ismale,isasian,iswhite,isblack,intersec_label,spe_label
/path/to/cropped_images/imgxx.png, 1(fake), 1(male)/-1(not male), 1(asian)/-1(not asian), 1(black)/-1(not black), 1(white)/-1(not white), 0(male-asian)/1(male-white)/2(male-black)/3(male-others)/4(female-asian)/5(female-white)/6(female-black)/7(female-others), 1(Deepfakes)/2(Face2Face)/3(FaceSwap)/4(NeuralTextures)/5(FaceShifter)
...
|- realtrain.csv
|_ img_path,label,ismale,isasian,iswhite,isblack,intersec_label
/path/to/cropped_images/imgxx.png, 0(real), 1(male)/-1(not male), 1(asian)/-1(not asian), 1(black)/-1(not black), 1(white)/-1(not white), 0(male-asian)/1(male-white)/2(male-black)/3(male-others)/4(female-asian)/5(female-white)/6(female-black)/7(female-others)
...
|- fakeval.csv
|_ img_path,label,ismale,isasian,iswhite,isblack,intersec_label,spe_label
/path/to/cropped_images/imgxx.png, 1(fake), 1(male)/-1(not male), 1(asian)/-1(not asian), 1(black)/-1(not black), 1(white)/-1(not white), 0(male-asian)/1(male-white)/2(male-black)/3(male-others)/4(female-asian)/5(female-white)/6(female-black)/7(female-others), 1(Deepfakes)/2(Face2Face)/3(FaceSwap)/4(NeuralTextures)/5(FaceShifter)
...
|- realval.csv
|_ img_path,label,ismale,isasian,iswhite,isblack,intersec_label
/path/to/cropped_images/imgxx.png, 0(real), 1(male)/-1(not male), 1(asian)/-1(not asian), 1(black)/-1(not black), 1(white)/-1(not white), 0(male-asian)/1(male-white)/2(male-black)/3(male-others)/4(female-asian)/5(female-white)/6(female-black)/7(female-others)
...Generate test.csv according to following format:
|- test.csv
|- img_path,label,ismale,isasian,iswhite,isblack,intersec_label
/path/to/cropped_images/imgxx.png, 1(fake)/0(real), 1(male)/-1(not male), 1(asian)/-1(not asian), 1(black)/-1(not black), 1(white)/-1(not white), 0(male-asian)/1(male-white)/2(male-black)/3(male-others)/4(female-asian)/5(female-white)/6(female-black)/7(female-others)
...Before running the training code, make sure you load the pre-trained weights. We provide pre-trained weights under ./training/pretrained. You can also download Xception model trained on ImageNet (through this link) or use your own pretrained Xception.
To run the training code, you should first go to the ./training/ folder, then you can train our detector with loss flattening strategy by running train.py, or without loss flattening strategy by running train_noSAM.py:
cd training
python train.py You can adjust the parameters in train.py to specify the parameters, e.g., training dataset, batchsize, learnig rate, etc.
--lr: learning rate, default is 0.0005.
--gpu: gpu ids for training.
--fake_datapath: /path/to/faketrain.csv, fakeval.csv
--real_datapath: /path/to/realtrain.csv, realval.csv
--batchsize: batch size, default is 16.
--dataname: training dataset name: ff++.
--model: detector name: fair_df_detector.
For model testing, we provide a python file to test our model by running python test.py.
--test_path: /path/to/test.csv
--test_data_name: testing dataset name: ff++, celebdf, dfd, dfdc.
--inter_attribute: intersectional group names divided by '-': male,asian-male,white-male,black-male,others-nonmale,asian-nonmale,white-nonmale,black-nonmale,others
--single_attribute: single attribute name divided by '-': male-nonmale-asian-white-black-others
--checkpoints: /path/to/saved/model.pth
--savepath: /where/to/save/predictions.npy(labels.npy)/results/
--model_structure: detector name: fair_df_detector.
--batch_size: testing batch size: default is 32.
After testing, for metric calculation, we provide python fairness_metrics.py to print all the metrics. To be noted that before run metrics.py, adjust the input to the path of your predictions(labels).npy files, which is the --savepath in the above setting.
Change --inter_attribute and --single_attribute for different testing dataset:
### ff++, dfdc
--inter_attribute male,asian-male,white-male,black-male,others-nonmale,asian-nonmale,white-nonmale,black-nonmale,others \
--single_attribute male-nonmale-asian-white-black-others \
### celebdf, dfd
--inter_attribute male,white-male,black-male,others-nonmale,white-nonmale,black-nonmale,others \
--single_attribute male-nonmale-white-black-others \Please kindly consider citing our papers in your publications.
@inproceedings{Li2024preserving,
title={Preserving Fairness Generalization in Deepfake Detection},
author={Li Lin, Xinan He, Yan Ju, Xin Wang, Feng Ding, Shu Hu},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2024},
}