search

sjtuplayer / anomalydiffusion

[AAAI 2024] AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model
MIT License
114 stars 14 forks source link

Is "caption" meaningful for the model? #30

Open p890040 opened 4 months ago

p890040 commented 4 months ago

Hi there,

I've noticed that within the inputs of the model, the "caption" is randomly selected from the strings in ldm/data/personalized.py => imagenet_templates_small.

However, given that this selection process is random, I'm curious about the significance of this caption for the model. Since both training and inference processes utilize this caption as one of the inputs, it's likely to have an impact on the model's output. Therefore, I find it a bit puzzling and would like to confirm with you, if I may be misunderstanding something.

sjtuplayer commented 3 months ago

I think the caption do not matter. Since the caption number is limited. The captions in inference time have been used in the training time. So, it would not influence the results too much. It is originally designed for the generation diversity.