search

bmaltais / kohya_ss

Apache License 2.0
8.77k stars 1.14k forks source link

Stable Diffusion 3 #2588

Open jchook opened 3 weeks ago

jchook commented 3 weeks ago

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

AngryOldWang commented 3 weeks ago

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

+1

NeoLoger commented 3 weeks ago

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

+1

cassiebug23 commented 3 weeks ago

I'm wondering the same thing, waiting for kohya to support SD3 training!

DenRakEiw commented 3 weeks ago

+1 :)

filliptm commented 3 weeks ago

Bumpity

ximinjie commented 3 weeks ago

+1

protector131090 commented 2 weeks ago

when can i train dreambooth? i need 3 0 training pleeeeeaseeee!

5KilosOfCheese commented 2 weeks ago

Will it be possible to use kohya to train Stable Diffusion 3 LoRAs?

I think we should wait for kohya-ss to update the script first. Bmaltais only made the GUI.

oO0 commented 2 weeks ago

And we need merger and better captioner. Also motion lora training, VAE training, and converting 2B to 8B

Charmandrigo commented 2 weeks ago

It's too soon to ask for that dude

sashaok123 commented 2 weeks ago

https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sd3.md

Bocchi-Chan2023 commented 2 weeks ago

https://github.com/kohya-ss/sd-scripts/commit/e5268286bf90ddcc53ad1deb31aba857cfa967d5

paboum commented 2 weeks ago

kohya-ss/sd-scripts@e526828

Did you have a chance to run it? There are already some SD3 LoRAs on Civitai... One of which used https://github.com/bghira/simpletuner for training.

trillgifted commented 2 weeks ago

https://huggingface.co/blog/sd3#dreambooth-and-lora-fine-tuning

zengjie617789 commented 2 weeks ago

https://huggingface.co/blog/sd3#dreambooth-and-lora-fine-tuning

Can you got better results using official trainning scripts? I trained it with my custom datasets, but the results are bad.

oO0 commented 2 weeks ago

does scripts or Simpletuner works on Windows?

BenDes21 commented 2 weeks ago

do we have an estimation for the sd3 update guy's ?

x-CK-x commented 2 weeks ago

do we have an estimation for the sd3 update guy's ?

This thread: https://huggingface.co/stabilityai/stable-diffusion-3-medium/discussions/98 seems to suggest that it's not quite working with the desired results yet, but it is in the works.

5KilosOfCheese commented 2 weeks ago

do we have an estimation for the sd3 update guy's ?

Lets just wait for Kohya to get it working proper first. If https://github.com/kohya-ss/sd-scripts/pull/1374 is to be belived it requires quite bit work and rewriting the script.

So this repo will not get it, until Kohya has done it. This system is just a GUI for Kohya's scripts.

Just chill. It's summer, people who got life other than this, they might be on a vacation or busy with other stuff. This is also quite complex thing to pull off, and just making it work is not enough - you need to make it work in a way that produces results worth doing.

Meanwhile go work on your datasets, so you'll have them ready to go.

bmaltais commented 2 weeks ago

SD3 future is not bright. Most SAI staff have left and civitai has banned the model and any models that might be trained using images produced by SD3... so there might not be much future and interest to finetune SD3 now.

DenRakEiw commented 2 weeks ago

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.

As for the future of SAI, we'll see what happens there and if they can recover.

5KilosOfCheese commented 2 weeks ago

SD3 future is not bright. Most SAI staff have left and civitai has banned the model and any models that might be trained using images produced by SD3... so there might not be much future and interest to finetune SD3 now.

Unless people publish the datasets I think trying to enforce the rule of banning all models trained with images from SD3. However, I think setting up the utilities to do this is still worth the effort. Since there is nothing wrong with using the model for non-commercial purposes.

I been having fun playing around with https://huggingface.co/common-canvas/CommonCanvas-XL-NC It isn't as bad as you'd think. It isn't what SDXL1 or even SD3 can do, but it's just version 0.1. However it's a model which is "legitimate" legally.

But there are interesting alternative architecture and models also available.

But lets not pretend that we didn't see this coming. I predicted this week after SD1.4 released, I'm amazed it took this long for something to happen on the legal front.

bmaltais commented 2 weeks ago

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3.

As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

5KilosOfCheese commented 1 week ago

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3. As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

In the pull request on Kohya which I linked it seems like it is coming. However in https://github.com/kohya-ss/sd-scripts/issues/1370 it would seems that the current knowledge on how to do SD3 LoRA training doesn't yield good results.

So as I emphasised in this discussion earlier. The community is going to need some patience. The fact that the script does get update, added to this QUI, doesn't mean that it produces anything worth the effort. Like it took a while people to realise that you shouldn't mess with the two text encoders in SDXL. It took a fair while for LoRA training to get good on the other models and lets be honest: there are still optimisations and improvements being made.

People really should just chill. Fact is that there is a good established base of models, and information on training which currently will yield superior results, because they been established and figured out. Because even if Kohya updates tomorrow, and you update the gui day after, people will bitch and cry about how the quality is bad, or training is slow, or it takes too much VRAM... until it all gets figured out, optimised and solved.

BenDes21 commented 1 week ago

As I understand it, the Civitai ban isn't permanent; it's just in place until the Civitai lawyers have reviewed the license or until SAI changes a small part in the fine print. The primary reason for the ban is to protect the community from legal issues. I do think there's a lot of interest in the community to be able to fine-tune SD3 or at least train LoRAs for SD3. As for the future of SAI, we'll see what happens there and if they can recover.

If Kohya add training support to sd-scripts I will certainly try to add support for it in the GUI… but I am not holding my breath.

sd3 added in the training scripts :

https://github.com/kohya-ss/sd-scripts/pull/1374/commits/d53ea22b2a8366e6bc9f14aaeec057cd817f60d3