Closed weipipione closed 2 months ago
Hi @weipipione , compared to SD or SDXL, we are training Generative Network from scratch. We can fine-tune it on the pre-trained model, but that was not the goal of the work. In the case of EEG-Image, the dataset is very small (N-way K-shot), and during the project, we aim to train the network from scratch with stability and high fidelity.
@prajwalsingh Thanks for quick reply! But I still have some concerns. The CVPR40 dataset contains only 40 categories of images, with 50 images per category. Training a new StyleGAN on such a dataset may lead to overfitting. If the generative model can only produce these 40 categories of images, the diversity of the images is fixed. Therefore, using metrics that only focus on image quality and diversity, such as IS, to evaluate the results might not be appropriate. Moreover, the training images include many with plain backgrounds, which could cause overfitting and make the generated images very similar to the ground truth (GT) images, thereby narrowing the gap between the generated model's distribution and the real data distribution. This might result in better FID and KID scores. Are there better evaluation metrics that could be used in this context?
@weipipione, This is a really great question regarding evaluation metrics. Other than that, people nowadays use CLIP-based scores or LPIPS distances. Recently, I came across one work, "Few-shot Image Generation via Cross-domain Correspondence." The authors proposed Intra-cluster LPIPS, which measures the diversity and fidelity of generated images in a few-shot setting. You can check Section 4 of this work for more information, which is the case in EEG2Image.
@prajwalsingh Thanks!
Hi @weipipione , compared to SD or SDXL, we are training Generative Network from scratch. We can fine-tune it on the pre-trained model, but that was not the goal of the work. In the case of EEG-Image, the dataset is very small (N-way K-shot), and during the project, we aim to train the network from scratch with stability and high fidelity.