How to test on different datasets? (i.e. Celeb-DF, DFDC)

[Kent1029]

How to test on different datasets? (i.e. Celeb-DF, DFDC)

Now there is only extract_frames_ldm_ff++.py in the project.

Will the extract method of Celeb-DF and DFDC be released in the future?

Has anyone successfully tested it on other datasets?

[Nku-cs-dsc]

We tested our model in Celeb-DF and DFDC under the same inference setting with the FF++. At first, we sample 32 frames from the input video. And align faces in every frame based on landmarks which extracted by the landmark detector (Dlib). Finally, feed faces into the deepfake detector to indicate face forgeries.

[lijichang]

@Kent1029

Hello. I am also encountering some challenges while trying to download the needed datasets, specifically the DFDC and Celeb-DF datasets. Can you give me some help? Thank you very much.

Firstly, as I understand it, downloading the DFDC dataset requires registration through an AWS account. Despite multiple attempts, I have not been successful in completing this registration. I am unsure if there is an issue with the process or if the problem lies on my end. I did, however, stumble upon a similar dataset on Kaggle, the details of which can be found at this link: Kaggle Deepfake Detection Challenge Dataset ↗. Could you please confirm if this is the same dataset used in the work I am referring to? If not, could you guide me on how to obtain a download to the DFDC dataset?

Secondly, while trying to download the Celeb-DF dataset, I found two versions available - Celeb-DF v1 and Celeb-DF v2. Could you kindly clarify which version was used in the paper?

I would greatly appreciate any assistance or guidance you can provide on these matters. Your insights will be invaluable in advancing my work in this critical area.

Looking forward to your response.

[yuntianlong2002]

when testing with celeb dataset, were you using List_of_testing_videos.txt or the whole dataset? Also the face detection failed to detect faces for some video, would the test script correct handle that case?(ignore frames with no faces)

[Nku-cs-dsc]

when testing with celeb dataset, were you using List_of_testing_videos.txt or the whole dataset? Also the face detection failed to detect faces for some video, would the test script correct handle that case?(ignore frames with no faces)

Hi, We sample nearest frame with facial landmarks to replace the frames which can not detect faces.

[Gnonymous]

@Nku-cs-dsc Follow your set above, l just get 80.77 AUC on CDF-v2,using the weight you public.

[Nku-cs-dsc]

@Gnonymous hi, you can check your face alignment method. If the face alignment method is different from the method used in the training stage, the model's performance on cross-dataset evaluation will degrade.

[Gnonymous]

@Nku-cs-dsc Sorry to bother, l want to ask some question about the set of train data. I have read your two papers, 'Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization' and 'Explaining Deepfake Detection by Analysing Image Matching', while I processed the dataset according to the 'extract_frames_ldm_ff++.py' file in your Github repository (while dividing the dataset according to the official repository's train,json). I found that the result of the processing is only 1 frame image taken under each video, I don't know if it is reasonable here, but the data should be in line with the FST-Match. the result of the training is not satisfactory, the AUC is almost maintained at 75-85% not on the rise, to ask you how to set the dataset in order to train the model effectively.

[Gnonymous]

@Nku-cs-dsc I delivered an e-mail to a publicly available e-mail address, but there seems to be no response

[Nku-cs-dsc]

@Gnonymous As mentioned in the Experimental setting part of our paper, we sample 32 frames in each video and train our detector under such training dataset. Through visualizing the videos of training data, you can find that the video contains many noisy frames. Such as frames without any face, some fake frames that have not been manipulated by the deepfake method, and some frames extracted wrong face landmarks. The deepfake model can not achieve high accuracy when trained by this dataset. By sampling more frames in a video and filtering out the frames with wrong face landmarks, the deepfake detector will achieve higher performance.

[Nku-cs-dsc]

@Gnonymous If you have some questions, feel free to contact me through the new email address: dongshichao1996@gmail.com