search

invoke-ai / InvokeAI

InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
22.79k stars 2.35k forks source link

[enhancement]: SDXL Lightning #5789

Closed JPBarringer closed 6 months ago

JPBarringer commented 6 months ago

Is there an existing issue for this?

Contact Details

No response

What should this feature add?

SDXL Lightning is another fast ("real-time"), low-step approach, recently released by Bytedance. Models are appearing on Civitai already, including some major checkpoints, and it would be nice to have explicit support in InvokeAI.

More info on Huggingface

Alternatives

No response

Additional Content

Looks (elsewhere) like it requires the scheduler to be sgm_uniform and Euler sampler, cfg of 0. And steps need to match the model. I think this raises questions for the model manager / linear UI – can a model limit linear UI options to a specific selection or range? (But there are likely existing config option that I'm not aware of here.)

gogurtenjoyer commented 6 months ago

You can install and use this right now with HF Repo ID: Vargol/sdxl-lightning-4-steps. Alternately, you can just use the released (not a full model) SDXL lightning unet paired with the rest of the needed SDXL sub-models yourself.

JPBarringer commented 6 months ago

Thanks. I have been poking around with it with a decent but of success – just also a lot of uncertainty, web searching, and trial and error.

I should probably clarify what I mean by "explicit support" in the feature request: I mean in the linear UI. SDXL Lightning merges could probably be visually distinct from base SDXL in the model selector. They could default to the right steps and CFG, or even remove options that wouldn't work. At the moment, it takes a lot of research, trial and error to make this work – or, at least, a lot more than just "select model, type prompt (then tweak if needed)" which seems like the ultimate UX goal for the linear UI.

Lightning makes generation much quicker, and thus makes Invoke more user-friendly. But only if the defaults work out of the box.

hipsterusername commented 6 months ago

Default params are planned for model manager. Not fully opposed to splitting lightning into its own category, but we'll see whether that's necessary as usage expands (and other model architectures are released)