Why the model generates images with 'pink filter' after training in a relatively large dataset?

tencent-ailab / IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.

Apache License 2.0

4.5k stars 296 forks source link

Why the model generates images with 'pink filter' after training in a relatively large dataset? #290

Closed sherrydoge closed 2 months ago

sherrydoge commented 5 months ago

Thanks for the great work! I met a strange problem when I try to train a model based on IP-adapter-base-sdxl. Specifically, the input data will contain 1~4 images and is all padded to 4. The model performs well when training with ~10000 samples, but generates images with 'pink filter' when training with ~100000 samples. I wonder if you have met this issue before or know what is happening in this situation. Looking forward to your reply!

xiaohu2015 commented 5 months ago

I haven't encountered similar problems. Maybe the size of training dataset is too small?

masaisai111 commented 2 months ago

What are your training parameters

sherrydoge commented 2 months ago

What are your training parameters

They are nearly the same with IP-Adapter. I have solved this issue, which is supposed to be caused by training dataset and overfitting.