10Ring / LAA-Net

The official implementation for LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection
Other
65 stars 9 forks source link

Regarding the test_image task, the results are bad. #1

Open cyh-del opened 4 months ago

cyh-del commented 4 months ago

Hello, author. I'm having some problems running the test_sbi.sh script as follows: 1.The test.py file is not normalised at line 123: label_pred = cls_outputs.cpu().numpy() Then I changed the code to : label_pred = _sigmoid(cls_outputs).cpu().numpy() In the efn4_fpn_sbi_adv.yaml file, I set subtask=test_image,set FROM_FILE in TEST to False, and use the pre-trained weights of using SBI(PoseEfficientNet_EFN_hm10_EFPN_NoBasedCLS_Focal_C3_256Cst100_8SBI_SAM(Adam)_ADV_Era1_OutSigmoid_1e7_boost500_UnFZ_model_best.pth). Test results were 94% accurate on 900 FF++ real images, but less than 50% accurate on 1205 FF++ fake images. These images are cropped to contain only faces. 2.Code in package_utils/cam_vis.py at line 103: scores = model(img_tensor.unsqueeze(0))[0]['cls'].sigmoid() Is this score equivalent to the label_pred? I commented out the CAM-related statements in the code to get only the prediction score for the image. But the test results using the dataset as above are still unsatisfactory. This may be some parameter setting error or some process error, maybe the test image still need to go through some pre-processing,
can you please help me to solve these problems or tell me some more detailed test process, looking forward to your reply, thank you!

10Ring commented 4 months ago

Hi @cyh-del , thanks for your interest in LAA-Net.

Thanks also for the comment on the line 123, you are right. We will fix it soon. We support the test_img only for a single image test. In case you want to evaluate LAA-Net on a custom dataset, please consult our preprocessing steps strictly. In our testing, with SBI we load the data from the preprocessing faces and additionally crop a 15% margin before running the evaluation. The current lines 100 and 101 are for single image inference with BI pretrained weights. We will update the docs for more transparent. Please try to have a look at our dataloader for more flexible. Many thanks.

ShikangWang commented 3 months ago

Hi @cyh-del , thanks for your interest in LAA-Net.

Thanks also for the comment on the line 123, you are right. We will fix it soon. We support the test_img only for a single image test. In case you want to evaluate LAA-Net on a custom dataset, please consult our preprocessing steps strictly. In our testing, with SBI we load the data from the preprocessing faces and additionally crop a 15% margin before running the evaluation. The current lines 100 and 101 are for single image inference with BI pretrained weights. We will update the docs for more transparent. Please try to have a look at our dataloader for more flexible. Many thanks.

Hi, @10Ring, thanks for your great work. I also have the same question about testing single image with SBI. Can you point out the exact position of the data perprocess codes when using SBI?

maryam-arani commented 2 months ago

Hi, this is a very interesting work! Did anyone manage to reproduce the paper AUC on celeb-df-v2 using pre-trained SBI model?

sheep1222 commented 2 months ago

Hello, I am a beginner and this code is a bit complicated for me. I would like to ask, which part of the E-FPN code is it?