search

yoctta / multiple-attention

The code of multi-attention deepfake detection
241 stars 54 forks source link

Inconsistent parameters between paper and code #8

Open misscc320 opened 3 years ago

misscc320 commented 3 years ago

Hi, thank you for your sharing code. I really appriciate your excellent work. I have read both the paper and your released code, but I found that some parameters described in the paper and that in the config setting of your code are inconsistent. For example, the batch size is set to 48 in the paper but 16 in the config code, the learning rate is start from 0.001 in the paper but 0.0001 in the config code, and so on. Since I want to correctly re-implement your method, I wonder which parameter I shoud use? By the way, I started the training from the pretrain() method of main.py, am I right?

yoctta commented 3 years ago

the batch size is set to 48 in the paper but 16 in the config code, because we use distributed data parallel. Learning rate set as 1e-4 is OK. And a little trick : after pretraining, train multi-attention with backbone froze and set AGDA loss to 0 for a few epochs.

From: misscc320 Sent: Tuesday, August 10, 2021 10:58 AM To: yoctta/multiple-attention Cc: Subscribed Subject: [yoctta/multiple-attention] Inconsistent parameters between paper andcode (#8)

Hi, thank you for your sharing code. I really appriciate your excellent work. I have read both the paper and your released code, but I found that some parameters described in the paper and that in the config setting of your code are inconsistent. For example, the batch size is set to 48 in the paper but 16 in the config code, the learning rate is start from 0.001 in the paper but 0.0001 in the config code, and so on. Since I want to correctly re-implement your method, I wonder which parameter I shoud use? By the way, I started the training from the pretrain() method of main.py, am I right? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

misscc320 commented 3 years ago

the batch size is set to 48 in the paper but 16 in the config code, because we use distributed data parallel. Learning rate set as 1e-4 is OK. And a little trick : after pretraining, train multi-attention with backbone froze and set AGDA loss to 0 for a few epochs. From: misscc320 Sent: Tuesday, August 10, 2021 10:58 AM To: yoctta/multiple-attention Cc: Subscribed Subject: [yoctta/multiple-attention] Inconsistent parameters between paper andcode (#8) Hi, thank you for your sharing code. I really appriciate your excellent work. I have read both the paper and your released code, but I found that some parameters described in the paper and that in the config setting of your code are inconsistent. For example, the batch size is set to 48 in the paper but 16 in the config code, the learning rate is start from 0.001 in the paper but 0.0001 in the config code, and so on. Since I want to correctly re-implement your method, I wonder which parameter I shoud use? By the way, I started the training from the pretrain() method of main.py, am I right? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Hi, thank you for your kind response. I read your response but also have some questions.

  1. Does the correct training process is call pretrain() first then call aexp()?
  2. If the first question holds, then the backbone is not frozen and the AGDA loss is not 0 with the configuration of aexp(). This is consistent with your replied trick. So which training process should I use?
  3. According to the code of pretrain(), this function only train the Efficientnet-b4 without attention block, texture enhance block and AGDA. If follow the trick mentioned in your response, then when to use AGDA?
damengdameng commented 2 years ago

hi, same confusions! Have you reimplemented this work successfully?