openai / consistency_models

Official repo for consistency models.
MIT License
6.02k stars 409 forks source link

Is the sampling reversible? (Image to Noise) #15

Open discordance opened 1 year ago

discordance commented 1 year ago

Hello! Your work is truly remarkable. I've had the opportunity to experiment with standard DDPM models using DDIM sampling, and despite being relatively slow, they exhibit a fascinating reversibility property. This allows denoising, as well as the reverse process, recreating the exact same input noise (or close).

I am seeing that there is two samplers being used here: sample_onestep and stochastic_iterative_sampler.

stochastic_iterative_sampler adds stochastic noise between each steps so is obviously not reversible. Not sure about sample_onestep.

I would be grateful if you could provide some insights into whether this reversibility property is preserved in Consistency models sampling. The examples provided do not explicitly demonstrate this aspect, and I am eager to learn more about it. Thank you for your time and expertise.

zheng-peng-fei commented 1 year ago

I'm equally interested in this issue. Have you implemented the reverse process? I'm also curious why this issue was closed?

discordance commented 1 year ago

Nope, no success here so far. Basically, what I am trying to achieve is two fold:

It's obvious how to do that once you can get from Image -> Noise because all those operation can be performed into the gaussian space and denoised again. But from what I understand, those models are learning to denoise directly, they are not anymore a score function that can be reversed at each iteration.

I have tried (at sampling time):

Edit: I closed the issue because I thought my question was premature but actually many people are looking for similar thing

krNeko9t commented 6 months ago

Nope, no success here so far. Basically, what I am trying to achieve is two fold:

  • Interpolation between two images using the model.
  • Take an image and generate infinite variations around.

It's obvious how to do that once you can get from Image -> Noise because all those operation can be performed into the gaussian space and denoised again. But from what I understand, those models are learning to denoise directly, they are not anymore a score function that can be reversed at each iteration.

I have tried (at sampling time):

  • Take noise X_t, denoise it into x0, interp x0 with the other image into x1 and go one iteration more into the model with x1. The results are not excellent.

Edit: I closed the issue because I thought my question was premature but actually many people are looking for similar thing

Hi, i'm looking for this problem's solution, are you or someone else have any idea? cause you say many people are looking for it, where should i find some place where i can discuss it?