search

HaiyuWu / BA-test-dataset

The official repository of "What Should be Balanced in a "Balanced'' Dataset?" at BMVC2023
0 stars 0 forks source link

Are there any methodological errors? #2

Open sydddl opened 3 weeks ago

sydddl commented 3 weeks ago

Sorry to bother you again. I'm not sure if you've read this new article:

Analyzing the Feature Extractor Networks for Face Image Synthesis

Snipaste_2024-08-27_18-14-12

You can see here that the Arcface model's focus on the face is right here on the hairline. But other models are not.

Does this mean that by tuning the dataset for a single model, even if the performance is balanced on that model, it cannot be used as a measure of the fairness of other models?

And when I cut the BA-test data set as the training set to train the model, I cannot obtain balanced performance in the i.i.d data set.

HaiyuWu commented 3 weeks ago

Hi, no worries about pull any issues. Thanks for sharing this paper. I would say that the synthetic images are kinda different from the real images. This article shows the attention map on real images: https://arxiv.org/pdf/2112.09786.

As for the BA-test dataset, it is designed for testing rather than training. I am not sure how the performance would be if it is used for training.

sydddl commented 3 weeks ago

Thanks，I ignored that they used generated data.

HaiyuWu commented 3 weeks ago

No, problem! Btw, if you are interested in synthetic dataset generation. You can check my recent work: https://arxiv.org/pdf/2409.02979