Sanity Check

[hosseinaskari-cs]

I'm curious to understand why, when I comment out the projection step in the code, it fails to function as DDIM sampling and generate any output. It's puzzling why the sanity check isn't working. I am not getting any errors, but rather I am getting the following images for inpainting task where I removed the line related to the projection, i.e., converting x0_t_hat = x0_t - lambda_t*Ap(A(x0_t) - y) to x0_t_hat = x0_t.

[wyhuai]

I think the operator may have problems. You can check if A(x) and Ap(A(x)) can generate reasonable results. By the way, which picture is the input, and which is the output? Can your model generate desired pictures without using DDNM?

[hosseinaskari-cs]

Thanks for your reply.

But we are not using an operator at all. As I mentioned I converted this converting x0_t_hat = x0_t - lambda_t*Ap(A(x0_t) - y) to this one x0_t_hat = x0_t. In this case, we are sampling unconditionally or randomly. Both pictures are samples or outputs.

[wyhuai]

That‘s weird, did you use your own pretrained model? Can your own pretrained model work on other diffusion backbones, e.g., DDPM?

[hosseinaskari-cs]

Not actually. I used the model you provided the link for, which is the same as the guided diffusion code. You can easily check your code. Just remove the inversion part ( x0_t_hat = x0_t - lambda_t*Ap(A(x0_t) - y) to this one x0_t_hat = x0_t.), then you get the same result.

[LiRunyi2001]

I've checked the output of cases without the projection step on ImageNet dataset, and some of them does look weird. I will check the code and pretrained model later.

[LiRunyi2001]

As the output has no direct relationship with proposed Zero-Range Decomposition, I guess it is likely the problem of pretrained model.

[LiRunyi2001]

I re-write the code of loading & inference (sampling) of pretrained DPM model from OpenAI, and it possibily does not have enough generating ability to sample good images from pure noise with no guidance. Besides, I have also tested the generating effect of model pretrained on CelebA-HQ dataset, and it is good. Test results of pretrained model from OpenAI:

Test results of model pretrained on CelebA-HQ dataset: