Results on GenImage

[krishnasrikard]

Hi

Thank You for the amazing work.

I have a small query regarding the results.

I observed an inconsistency in the numbers reported by GenImage [1] and GenDet [2] and your work [3] while training and validating models on the GenImage dataset. Your numbers are much higher than the numbers reported by either GenImage or GenDet.

Could you please provide a clarification? What is the difference in training? Or are the numbers reported by GenImage authors a miscalculation/bug?

Extremely sorry for the trouble.

Thank You.

[1] Zhu, M., Chen, H., Yan, Q., Huang, X., Lin, G., Li, W., Tu, Z., Hu, H., Hu, J. and Wang, Y., 2024. Genimage: A million-scale benchmark for detecting ai-generated image. Advances in Neural Information Processing Systems, 36.

[2] Zhu, M., Chen, H., Huang, M., Li, W., Hu, H., Hu, J. and Wang, Y., 2023. Gendet: Towards good generalizations for ai-generated image detection. arXiv preprint arXiv:2312.08880.

[3] Chen, B., Zeng, J., Yang, J. and Yang, R., DRCT: Diffusion Reconstruction Contrastive Training towards Universal Detection of Diffusion Generated Images. In Forty-first International Conference on Machine Learning.

[beibuwandeluori]

Thank you for your interest in our paper. We apologize for the missing details in the experiments section. In the experiments on GenImage in our paper, all comparison methods were reproduced using my training scripts. For example, in Table 2, we only used the "stable_diffusion_v_1_4/imagenet_ai_0419_sdv4" subset for training. During training, we divided the images in the "train" directory, using 90% as the training set and the remaining 10% as the validation set. Finally, we used the images in the "val" directory as the final test set.

[krishnasrikard]

Hi

Thank you very much for the swift reply. Apologies for my delayed reply.

The authors of GenImage and GenDet also trained on the SDv1.4 subset of GenImage (i.e. both real and fakes from GenImage dataset {reals from ImageNet and fakes generated from SDv1.4}).

You mentioned that all comparison methods were reproduced by my training scripts. Do you mean that during training of prior works (assuming that the real and fake images are taken from GenImage SDv1.4 subset), you employed the data augmentation procedure mentioned in the paper?

Please let me know if I missed or misunderstood anything.

Thank You.

[beibuwandeluori]

Yes, both real and fake images are taken from GenImage SDv1.4 subset. In addition, the data augmentation used has been open sourced in the code "data/transform.py" (maybe it is not clear enough in the paper, you can check my source code for more details).

[krishnasrikard]

Thank you very much for the clarification.