search

apple / ml-stable-diffusion

Stable Diffusion with Core ML on Apple Silicon
MIT License
16.46k stars 886 forks source link

SDXL Turbo support? #305

Open pj4533 opened 7 months ago

pj4533 commented 7 months ago

Not sure what is required, but I thought I'd add an issue for discussion? I would love to integrate this into my project. Could be a huge leap forward.

https://huggingface.co/stabilityai/sdxl-turbo

RageshAntonyHM commented 7 months ago

Asked the same in Moshi Diffusion https://github.com/godly-devotion/MochiDiffusion/issues/348

cchance27 commented 6 months ago

Turbo just works to my knowledge, i've tried it using the comfyui node and it works and that projects just pulling in this project for the backend

pj4533 commented 6 months ago

Turbo just works to my knowledge, i've tried it using the comfyui node and it works and that projects just pulling in this project for the backend

I converted it myself, and could get image generation working by removing the refiner (which I guess is optional), but saw no improvement in number of steps required to generate an image, so I don't think it just works. I asked in the MochiDiffusion discord and was told: "SDXLTurbo needs a turbo-specific noise scheduler and an Euler ancestral sampler to work, neither is available at the moment with ml-stable-diffusion"

I am way more a front-end coder, and don't understand the inner workings of the SD models, so not really sure how to interpret that quote, but maybe someone else can comment?

operatesspaceships commented 5 months ago

We have a working implementation over at Odyssey. Essentially, what you need to do is to grab the LCMScheduler class from GuernikaKit (massive kudos to them, by the way), massage it a bit to fit the diffusion pipeline class found on this repo, set the timeStepSpacing to trailing, and then use a guidance scale of exactly 1 to run a Turbo model. Feel free to hit me up if anyone needs more detailed guidance.

nosferatu500 commented 4 months ago

I don't want to spam here, but in case anyone else wants to implement SDXL Turbo support in his/her app, all you need to do it's add SDE or LCM schedulers support and you're ready to go. I implemented it in my own app (DreamFusion, completely free), if you interested feel free to try it out!

PS: DreamFusion also support v-predict model (Terminus XL Gamma v2 beta1), turbo models (2.1 turbo, DreamShaperXL turbo), inpaint, instruct and other things. I basically made it for my self because of lacks some funcs I needed it ml-stable-difusion around 4 month ago but decided to share it with others and it's free!