several researchers and practitioners have found that when trained with the same “random seed” leads to similar images (Nichol et al., 2022)

[JunMa11]

Dear @ChenWu98 ,

Given two stochastic DPMs G1 and G2 that model two distributions D1 and D2, several researchers and practitioners have found that sampling with the same “random seed” leads to similar images (Nichol et al., 2022)

For the above claim, would it be possible to point out the corresponding results in this paper? https://arxiv.org/pdf/2112.10741.pdf

It seems that all the compared models are trained on the same domain in the GLIDE paper.

[ChenWu98]

Hi, if you compare Figure 12(a) with Figure 12(b), and Figure 13(a) with Figure 13(b), several pairs of images with the same random seed are very similar to each other. This is the earliest observation of the phenomenon we can find, so we cited this paper.

In our claim, we avoided using "trained on two distributions"; instead, we say that "G1 and G2 that model two distributions". Specifically, when conditioned on two different texts (e.g., "toys for boys" and "toys for girls" in Figure 12), the two conditioned diffusion models model two different but similar distributions; when using different guidance scales (e.g., 0.0 and 3.0 in Figure 13), the two guided diffusion models model two different but similar distributions.

[JunMa11]

Thanks for your reply very much.

I'm still confused about the two diffusion models.

1. in fig 12 and 13, you mean the two diffusion models have the same trained weights but different conditions, right?
2. For unpaired image-to-image translation, the two models have different weights (e.g., dog and cat models). Given the same seed, can they generate similar results? The answer could be no. in this background, how to understand the same “random seed” leads to similar images?

[ChenWu98]

1. It depends on how you define “weights”. I view “weights” as nodes on the computation graph that are independent of the noises, so it is quite flexible. Specifically, given two different text prompts, if you view the text encoder and the image generator as “weights”, then they have the same weights with different conditions; however, if you view the encoded text and the image generator as “weights”, then they have different weights (but sharing part of the weights, i.e., the image generator).
2. Our observation is that, using the same noise schedule, a cat diffusion model and a dog diffusion model can generate similar images with the same random seed. This is not guaranteed, but we did observe a certain level of similarity.