madebyollin / maple-diffusion

Stable Diffusion inference on iOS / macOS using MPSGraph
https://madebyoll.in/posts/maple_diffusion/
MIT License
793 stars 51 forks source link

Create Swift Package #2

Closed mortenjust closed 1 year ago

mortenjust commented 1 year ago

Thanks for this amazing work! This truly feels next level.

I saw your vision about MD being a backend to other apps, so I thought a good first step would be to turn it into a Swift Package. Here's the developer experience I envision

Easy to integrate. The simplest implementation could be:

  1. Developer adds the dependency to their project
  2. Add MapleDiffusion as an observed object on a SwiftUI view, giving it two optional arguments: localModelURL, remoteModelURL
  3. If it doesn't find the bins folder in the bundle or in the localModelURL, it starts downloading the files from the remoteModelURL and unzips to the localModelURL destination

That means a few substantial changes to this codebase, so you may not be interested in merging into this repo, but I am of course open to it. Here's what I did so far

To not have the app freeze the first 10 seconds, it is now loading the model asynchronously, showing the UI immediately

I also created a Combine Publisher wrapper for the callback-based generate function in the main class. Calling it, a developer gets a publisher they can pass around between pieces of their app and listen to changes.

I made a few small adjustments to the example app

My WIP fork is here https://github.com/mortenjust/maple-diffusion/

madebyollin commented 1 year ago

Thanks Morten! I tried your fork and it's definitely an improvement :)

You're probably going to be able to make much faster progress developing it in a fork / new repo, honestly (I'm not that invested in maintenance & don't know Swift well enough to efficiently review PRs anyway). I'm happy to link to your fork in the README here.

One UI note regarding the seed + randomizer - I suspect that the interface might work better as "seed input + lock icon", such that the lock lets you freeze & iterate on the current seed, and unlocking automatically reseeds whenever you click Generate. Seed + reset button compels the user to hit an extra tiny click target before each image generation, which seems less fun than just mashing on the one big "Generate" button.

mortenjust commented 1 year ago

Great, thanks for letting me know - I'll keep going in my own fork/repo and make sure to backlink and credit

I think you're right about the lock UI, that's an elegant way to do it, and more aligned with real use