search

madebyollin / maple-diffusion

Stable Diffusion inference on iOS / macOS using MPSGraph
https://madebyoll.in/posts/maple_diffusion/
MIT License
793 stars 51 forks source link

Variable Image size #11

Open divamgupta opened 1 year ago

divamgupta commented 1 year ago

Right now the trivial way to generate images of different sizes is to recreate the Maple Diffusion object with the new size params. Is there any way which is more efficient?

madebyollin commented 1 year ago

Per Apple's WWDC session / docs it should be possible to support variable image sizes efficiently:

  1. Replace the height / width usage in UNet placeholders with -1 to indicate an unspecified dimension
  2. Set a flag to skip type inference during UNet graph compilation

I started the code to do this in a branch here: https://github.com/madebyollin/maple-diffusion/commit/f590ea8767c95023784f1a1d684f107e32eec7b6. Currently it fails to compile the graph, though.

I think it's probably going to be possible-but-annoying to get variable shapes working - since it requires editing the graph code in order to make sure everything compiles (this may require adding some graph.shapeOf operators when reshaping an hw dim into separate h and w dims, for example, or just never creating any hw dims).

Gitterman69 commented 1 year ago

Diffusion Bee integrated the approach in a perfect way!!!

ShirasawaSama commented 7 months ago

@madebyollin Hello, is there any new progress on this issue?

divamgupta commented 7 months ago

I actually implemented it with the help of @madebyollin . Code is a bit messy I can make it public or make a PR some day

ShirasawaSama commented 7 months ago

This is good news, and I can also help you organize the code if you need it.