gan-police / frequency-forensics

Deepfake detection using wavelet-packets in PyTorch, European Conference on Machine Learning (ECML PKDD) 2022.
Other
47 stars 9 forks source link

Packet size does not match the model #25

Closed zjbthomas closed 2 years ago

zjbthomas commented 2 years ago

Hi there, thanks for sharing the code for such a great work!

I am trying to train the network using my own dataset and I want to use a similar setup as CNN-ln-db4 in your paper.

I prepared the dataset using the following commends (all paths are omitted):

python -m freqdect.prepare_dataset --log-packets --wavelet db4 --level 3

My dataset contains images all in size 128 x 128.

And I run the training script as following:

python -m freqdect.train_classifier \
--data-prefix log_packets_db4_reflect_3 \
--features packets \
--nclasses 8 \
--model cnn

However, I got the following errors at line 147 of models.py.

RuntimeError: mat1 and mat2 shapes cannot be multiplied (512x1176 and 24x8)

By checking the code, I think this is because the input packet size is bigger than what is expected, so after the convolutions the output does not go down to 1 x 1.

I am wondering did I make any mistakes in preparing the dataset? What should I do to have packets with the correct size?

Thank you.

v0lta commented 2 years ago

Okay, I see. You are using padded wavelets. Reflection padding adds extra values on the edges to make the wavelet transform invertible. We did not do that and chose to work with boundary wavelets instead. Since extra padding adds extra pixels the dimensions don't work. However, https://arxiv.org/pdf/2210.14874.pdf (Table 2) evaluated padding and found it to work well. Therefore you should be able to safely adjust the hardcoded 24 in line https://github.com/gan-police/frequency-forensics/blob/463458f3f4016b120325423a8fcd198d225aefb8/src/freqdect/models.py#L47 .

v0lta commented 2 years ago

An other option would be to run prepare_dataset.py with --boundary boundary.

zjbthomas commented 2 years ago

Thanks, it works!