search

liuliu / swift-diffusion

BSD 3-Clause "New" or "Revised" License
423 stars 33 forks source link

Why are there two UNets in SDXL? #51

Closed ghost closed 9 months ago

ghost commented 9 months ago

Why is there a UNet and Unet_fixed? Is that some optimization to save memory?

ghost commented 9 months ago

Its an interesting idea, that you pre-compute the keys and values of the next encoder part. But does that give any significant memory gains?

liuliu commented 9 months ago

It saves about 600MiB RAM (Float16, less if it is 8-bit model). Computation saving is negligible (you don't need to compute these on every iteration, hence computation saving).