search

crowsonkb / k-diffusion

Karras et al. (2022) diffusion models for PyTorch
MIT License
2.26k stars 372 forks source link

Question: Is it possible to predict step count of DPM adaptive? #36

Closed C43H66N12O12S2 closed 1 year ago

C43H66N12O12S2 commented 1 year ago

At https://github.com/AUTOMATIC1111/stable-diffusion-webui we use learned conditioning which just replaces the fed prompt at a specified step count [prompt1:prompt2:0.5] would switch at %50 of the step count for example.

Since DPM adaptive picks its own step counts, this feature cannot work accurately. Is there a way we could predict the step count of DPM adaptive for any given prompt so we could integrate it to this feature?

crowsonkb commented 1 year ago

I would recommend switching over at a particular noise level instead of a particular step count. For instance say your sigma_max (starting noise level) is 80 and your sigma_min (ending noise level) is 0.01. Then in log domain you are going from 4.382 to -4.605, and 50% of the way through is -0.11157, or sigma=0.89442. DPM Adaptive uses -log sigma as its time variable internally so this makes sense to do for it. If you use a callback you can get the noise level (sigma) for a step.

Thank you, Katherine Crowson