It's hard to understand and use this dataset

[chandlerbing65nm]

Dataset used in confusing.

You did not indicate what real image is used in a particular DM images.

In thiis folder structure:

Testset directory
|--biggan_256
|--biggan_512
.
.
.
|--real_coco_valid
|--real_imagenet_valid
|--real_ucid
.
.
.
|--taming-transformers_segm2image_valid

what real dataset is paired with [biggan_256, biggan_512, ..., taming-transformers_segm2image_valid]

[RCorvi]

I apologize for the late reply, unfortunately Github is not sending me emails for new issues opened on this repository. In our case, the folders for the same architectures are pooled together for example in the case of BigGAN the metrics are calculated considering biggan_256 and biggan_512 at the same time. As for the real images, all of them are used for every dataset and rather then using the accuracy score we use the balanced accuracy score. If you need anything else let me know.

[chandlerbing65nm]

@RCorvi Could you clarify how the balanced accuracy score is computed when pooling multiple real datasets?

Do you mean you used three different real datasets - IMAGENET, UCID,COCO - Common Objects in Context - for every fake folder, then averaged them?

Also, can you indicate the subsets of the IMAGENET, UCID,COCO - Common Objects in Context you used? and how many each?

[RCorvi]

We use the three real datasets for every fake dataset. Therefore, the metrics is computed considering for real data all three of them and for fake data all the folders of that dataset. In the metrics_evaluation.py file you can see how the folders are pooled for each fake dataset, all the folders with a certain architecture in the name are pooled together. The balanced accuracy is equal to the arithmetic mean of sensitivity and specificity, in our case we use the algorithm present in scikit learn metrics. As for the real images used, they are noted in the csv file present in the zip file.