FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection

This is a Haiku & Jax implementation of the FakeOut paper.

The figure above describes the two parts of FakeOut. On the left is the pre-training out-of-domain self-supervision phase, via MMV. On the right is the adaption phase of FakeOut to the video deepfake detection task.

Setup

Arrange the videos of the desired dataset in train, val, test dedicated directories. Run our face tracking pipeline as documented in the FacePipe repo.

To use FakeOut, first clone the repo:

cd /local/path/for/clone
git clone https://github.com/gilikn/FakeOut.git
cd FakeOut

Install requirements

pip install requirements.txt

Prepare your dataset using the script:

python fakeout/data/data_preparation.py
    --dataset_name {face_forensics, dfdc, deeper_forensics, face_shifter, celeb}
    --split {train, test}
    --videos_path /path/to/facepipe/output

Our checkpoints are made available in the following links:

FakeOut TSM-50x2 (V&A)

For inference, execute the following script (DFDC test-set running example):

python fakeout/inference.py
    --checkpoint_path /local/path/to/checkpoint
    --dataset_name {face_forensics, dfdc, deeper_forensics, face_shifter, celeb}
    --use_audio {True, False}
    --mlp_first_layer_size {6144 (for TSM-50x2 (V&A)), 4096 (for TSM-50x2 (V))}
    --num_test_windows {e.g. 10}

For fine-tuning, execute the following script (FF++ train-set running example):

Will be availabe soon...

BibTex

If you find FakeOut useful for your research, please cite the paper:

@article{knafo2022fakeout,
  title={FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection},
  author = {Knafo, Gil and Fried, Ohad},
  journal={arXiv preprint arXiv:2212.00773},
  year={2022}
}