[Feature Request]: Stable Diffusion 3 Medium support

[bendikv]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

Support Stable Diffusion 3 Medium https://huggingface.co/stabilityai/stable-diffusion-3-medium

Proposed workflow

Just like other SD models

Additional information

No response

[PierrunoYT]

I need it now

[gorilla-san]

I need it yesterday

[rt974]

i will need it last week

[PythonLimited]

i need it last month

[grubeninspekteur]

Don't spam please... like you guys I'm subscribing to this to be notified when it gets resolved, not for "+1" comments 😉

[alessandro893]

[amarand]

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

[AhmedAKILI]

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

SD officially supports comfy but not a1111. actually One of their employees develops comfy so they use it internally kinda

[light-and-ray]

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

The comfy Dev works in Stability AI. He had this support and weights last months, but wasn't showing to us

[amarand]

The comfy Dev works in Stability AI. He had this support and weights last months, but wasn't showing to us

Yeah, I posited a similar question over on Reddit and was told that ComfyUI is "the official" UI for Stability. What's funny is, every time I try to use ComfyUI, it makes me want to cry, and I'm a computer guy (had to rhyme)...with decades of experience. ComfyUI is just not a user friendly experience.

[amarand]

The comfy Dev works in Stability AI. He had this support and weights last months, but wasn't showing to us

That's an unfair advantage that makes me want to support AUTO1111 even more! 😹

[roelfrenkema]

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

Probably because SAI decided to use Comfy as their prefered backend. So its not really a ready on day one issue there

[AG-w]

Still don't work for me Stable Diffusion 3 medium

please scroll up and read

[CommanderCRM]

Despite some TG channels saying that it works with SD3 Medium, it doesn't (for those who might try and waste time now):

• fails to load (RuntimeError: Error(s) in loading state_dict for LatentDiffusion)
• fails to generate ( AttributeError: 'NoneType' object has no attribute 'lowvram')

[wfjsw]

This time it is even worse as SGM (provided by Stability for the backend of SDXL) is not getting an update and the comfyui one is viewed as official implementation. I believe AUTOMATIC1111 is working on it at the moment but as there are major changes it may take a few days.

[light-and-ray]

I have added SD3 support in my extension sd-webui-old-sd-firstpasser in advance 😛

I'm sure the developers will add sd3 support within a few days

[TheAntiCamper]

Much love. Take your time. Can wait as long as it takes to use the best UI.

[AbstractEyes]

Is it ready yet?

[Myoko]

Waiting for SD3 support

[EriIaz]

Waiting for SD3 support

This actually is a good representation of current SD3 performance, lol. The anatomy is so godawful it makes me wonder if the model is a train wreck or there's a deployment issue, so I am not sure ComfyUI implementation is alright.

[wfjsw]

The ComfyUI one is the official reference implementation by Stability.AI staff, so unfortunately

the model is a train wreck

[EriIaz]

The ComfyUI one is the official reference implementation by Stability.AI staff, so unfortunately

the model is a train wreck

Something being official reference doesn't mean it can't have a major error. For instance, LLaMA-3 models were initially coming with the wrong tokenizer configuration from Meta. Mishaps happen. The question is, if the model is serviceable without a retrain or not. It better be a bug in ComfyUI, because the model doesn't worth A1111's & co time otherwise.

I'd much rather take PixArt Sigma implementation that SD3 as it is right now.

[bigmover]

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

The comfy Dev works in Stability AI. He had this support and weights last months, but wasn't showing to us

I need to warning Stablity to face this issue that the WebUI has made an outstanding contribution to the prosperity of SD. It makes us angry：

[GreyWarden20]

The older version of webuinid much better compare with the newer version because honestly wr cannot worknit anymore. Iam stuck at the loading of models.

On Fri, Jun 14, 2024, 3:18 PM Jack @.***> wrote:

I totally understand if the answer is "the developer has limited time" but...I've noticed that other UIs support the new SD3 weights (for example ComfyUI) on drop day (today). Is there a deeper reason why AUTO1111 doesn't "just work" at the moment, while other UIs do?

The comfy Dev works in Stability AI. He had this support and weights last months, but wasn't showing to us

I need to warning Stablity to face this issue that the WebUI has made an outstanding contribution to the prosperity of SD. It makes us angry：

— Reply to this email directly, view it on GitHub https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/16006#issuecomment-2167388797, or unsubscribe https://github.com/notifications/unsubscribe-auth/BIELETOJUXAALUGBA4GK3IDZHKKLXAVCNFSM6AAAAABJGLXFYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRXGM4DQNZZG4 . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

[Myoko]

Waiting for SD3 support

This actually is a good representation of current SD3 performance, lol. The anatomy is so godawful it makes me wonder if the model is a train wreck or there's a deployment issue, so I am not sure ComfyUI implementation is alright.

the bad anatomy its prompt issues, i find use segmented prompt could fix this issue on civitai

[CamiloMM]

For anyone curious why WebUI gets thrown under the bus by Stability despite having been the entry point for many and arguably the UI for SD, Automatic1111 has made them angry in the past by implementing code that supported the NAI leaked model (a collaboration between NovelAI and Stability), which they even gaslit people into believing he stole leaked code (the code was public, he just added a line that allowed everyone to use the model that 4chan leaked).

For this and a few other reasons (such as ""problematic"" game mods he made in the past) Stability has deemed Automatic1111 persona non grata, and even banned him from their Discord when the leak happened. They go out of their way to highlight ComfyUI as the one and only UI.

[Nemesis-the-Warlock]

Waiting for SD3 support

This actually is a good representation of current SD3 performance, lol. The anatomy is so godawful it makes me wonder if the model is a train wreck or there's a deployment issue, so I am not sure ComfyUI implementation is alright.

the bad anatomy its prompt issues, i find use segmented prompt could fix this issue on civitai

Well i think they want to avoid the generation of a certain flavor of content and the model reflects this. There will be retrained models, of course... its a fairly good model otherwise, the best with text so far.

[protector131090]

Any news on A1111 support? i hate comfy...

[sinand99]

Honestly I would prefer the dev team to focus on performance improvements, since SD3 is total garbage now. They can take their time until there are proper fine tuned SD3 models, which would take months possibly.

[l0stl0rd]

Honestly I would prefer the dev team to focus on performance improvements, since SD3 is total garbage now. They can take their time until there are proper fine tuned SD3 models, which would take months possibly.

Or it might never get much support or use and end up like SD2. Which is my guess at the moment.

[Ryan-Haines]

Honestly I would prefer the dev team to focus on performance improvements, since SD3 is total garbage now. They can take their time until there are proper fine tuned SD3 models, which would take months possibly.

If A1111 doesn't support SD3, less people will fine tune SD3 models. Chicken and egg problem.

[protector131090]

Honestly I would prefer the dev team to focus on performance improvements, since SD3 is total garbage now. They can take their time until there are proper fine tuned SD3 models, which would take months possibly.

its not garbage. it cand do human but it can do things no other model can with landscapes, interior, textures and many other things. Quality is amazing.

[CamiloMM]

Honestly I would prefer the dev team to focus on performance improvements, since SD3 is total garbage now. They can take their time until there are proper fine tuned SD3 models, which would take months possibly.

Or it might never get much support or use and end up like SD2. Which is my guess at the moment.

Biggest problem with SD2 was not that it sucked (it sucked less than 1.5 sucked) but that SDXL sucked even less at launch. So people finetuned SDXL. In a world without SDXL, models like Animagine, Juggernaut and Pony would be SD2-based.

The actual problem with SD3 is that I assume you can't finetune it on a normal computer. You'll notice SD 1.5 has more LoRAs than XL because a lot of people can only make those, for example. I assume (hopefully I'm mistaken) the great finetunes would require renting out an H100 for a couple hundred hours? (at least they already have the dataset which is the hard part).

[Nemesis-the-Warlock]

According to documentation fine tuning should be possible on "consumer grade" hardware for the "medium" sized model, the unreleased large model probably not.

[CamiloMM]

According to documentation fine tuning should be possible on "consumer grade" hardware for the "medium" sized model, the unreleased large model probably not.

If the checkpoint fine-tuning requirements are any indication, maybe LoRA requirements are similar to SDXL which would be very good!

[EriIaz]

According to documentation fine tuning should be possible on "consumer grade" hardware for the "medium" sized model, the unreleased large model probably not.

If the checkpoint fine-tuning requirements are any indication, maybe LoRA requirements are similar to SDXL which would be very good!

I don't know if the LLM experience translates to this, but you can fine-tune an 8B model with 24GB using QLORA.

The issue is the its licensing. I am not a legal expert, but it's written like if you're earning a penny with SD3, you are supposed to negotiate a creator's licence, which in turn obligates you to destroy your derivate works whenever you terminate the agreement for any time or reason. So you must delete and stop providing your Loras. Not sure about generated content, but probably applies to it too. It's ridiculous, but this is how it is as I understand it. It also puts all the blame on the licensee instead of the end-user if you're providing it to the community and someone does something illegal.

I am actually not even sure if the backend code classifieds as derivative work or not. If that's the case, imagine burning this repo just because A1111 is a law abiding citizen and missed the payment.

[Nemesis-the-Warlock]

This is not for commercial use. And many models come with agreements, even the derivatives. How binding this is in actual law? I am not an attorney, i have no idea. What i know is that models are shared freely and used freely for the most part. We aren't talking about using these models commercially nor the derivative management. Simply a platform to run these and... hopefully train and merge them. And this is okay. This aside very few people actually start a company using these commercially and even if they do to... lets say print t-shirts, you'd first have to prove without a doubt a certain model was used in the creation of said t-shirt to apply its license IF it even holds water in court. So... uhm... i don't think its really a concern aside of pictures are used commercially online only by a company where the metadata is still embedded (which would be... well... stupid).

[Koitenshin]

People have already dumped the entirety of the model. It's definitely a trainwreck. And I'm not sure why it includes Pixart Sigma Alpha inside of SD3.

[EriIaz]

I have to apologize to Comfy for doubting his work in the light of this

Chances are, wfjsw was right.

[txiaou]

AUTO1111 加油！

[PierrunoYT]

Can you keep me updated with the latest progress related to SD3?

[tanggogogo123]

Can you keep me updated with the latest progress related to SD3?

已经在做了，分支中有SD的代码修改，但还未合并，你可以去看看

[bigmover]

Can you keep me updated with the latest progress related to SD3?

已经在做了，分支中有SD的代码修改，但还未合并，你可以去看看

Perhaps need a teamer ？ Have you noticed someone else is dong something similar to you?

[roelfrenkema]

Can you keep me updated with the latest progress related to SD3?

Why dont you just keep an eye on the SD3 Branch?