A tiny question in diffusion.py

[shoh4486]

Thanks for the nice work, and I am trying to utilize the diffusion in my GAN model. And I have a minor question in your code:

(diffusion.py)

        if self.ts_dist == 'priority':
            prob_t = np.arange(t) / np.arange(t).sum()
            t_diffusion = np.random.choice(np.arange(1, t + 1), size=diffusion_ind, p=prob_t)

You set prob_t as np.arange(t) / np.arange(t).sum(), not np.arange(1, t + 1) / np.arange(1, t + 1).sum(). According to the former case (your code), the first diffused image is skipped. If T is 10, prob_t is [0, 0.02222222, 0.04444444, 0.06666667, 0.08888889, 0.11111111, 0.13333333, 0.15555556, 0.17777778, 0.2], and if alphas_bar_sqrt is tensor([1.0000, 0.9999, 0.9988, 0.9965, 0.9932, 0.9887, 0.9832, 0.9766, 0.9690, 0.9603, 0.9507]) and one_minus_alphas_bar_sqrt is tensor([0.0000, 0.0100, 0.0491, 0.0832, 0.1167, 0.1498, 0.1826, 0.2151, 0.2472, 0.2790, 0.3103]), the second items in those tensors, that is, 0.9999 and 0.0100 are never drawn to the time step t.

Is this the intended behavior? or is there any misunderstanding of myself?

Thanks in advance, and again thank you for your nice work.

[Zhendong-Wang]

Hi there, good catch! I remembered that I tried both np.arange(t) / np.arange(t).sum() and np.arange(1, t + 1) / np.arange(1, t + 1).sum(), and the choice doesn't matter. The code here is with the first option. But you definitely could try your own scheduling options.

[shoh4486]

Alright. I appreciate your kind answer :) Have a nice day