Stable Diffusion- WebUI Implementation

[aria1th]

Thanks for the great work! I have few questions about it.

Short Implementation

https://gist.github.com/aria1th/04bb78207daeee1f3d0800dc422e6254 WebUI Implementation

We can understand it as caching the result, and use for nearby steps, skipping UNet blocks to use cache.

But, for DDIM and PLMS - I observed it works pretty well, but for other samplers, the changes are drastic for certain steps, such as DPM ++ SDE a Karras.

For those samplers, seems like we can stop caching for initial steps, focusing on 'stabilizing' steps, like stopping for 30-40% of initial steps, or invalidating cache if timestep is bigger than 50 between steps.

Is it correct?

Also, I tried implementation for controlnet too - but seems like it is conflicting, it naturally uses timestep dependent embedding, to schedule guidance...

Maybe some masked guidance / combined guidance is required?

[TingTingin]

just as a heads up this guy here also had an implementation that was of the same idea (though developed independtly of DeepCache) it may have some insights for your code https://gist.github.com/mcmonkey4eva/25f2b4d95a5ede0934885d74c0669293

[VainF]

Hi @aria1th, thanks for reaching out.

For those samplers, seems like we can stop caching for initial steps, focusing on 'stabilizing' steps, like stopping for 30-40% of initial steps, or invalidating cache if timestep is bigger than 50 between steps.

It's quite an interesting idea. I believe that designing a more smart and clever scheduler, like skipping some steps or non-uniform caching, should be helpful.

Also, I tried implementation for controlnet too - but seems like it is conflicting, it naturally uses timestep dependent embedding, to schedule guidance...

We don't have much experience in ControlNet. DeepCache is an initial work and might conflict with some techniques if the redundancy assumption is violated.