new SD3 branch train error

[GoofyAi]

Discussed in https://github.com/bmaltais/kohya_ss/discussions/2630

^{Originally posted by **asrafulalam02** July 8, 2024} hello i was trying the SD3 branch for testing sd3 training. it installed right and I got all the sd3 settings right. but when I start the training it would stop the training with an error of "AssertionError: training without text encoder cache is not supported currently / text encoderのキャッシュなしの学習は現 在サポートされていません" check it out:  but here are my settings on the ui and the toml config. as you can see i have the option enabled on both side but im still getting the error.   can anyone help me with it?

[bmaltais]

I have a feel you will get more help directly on the sd-scripts repo about the sd3 code. Kohya is still working on it... so it is quite possible it is not fully baked yet.

https://github.com/kohya-ss/sd-scripts/commits/sd3/

[GoofyAi]

I have a feel you will get more help directly on the sd-scripts repo about the sd3 code. Kohya is still working on it... so it is quite possible it is not fully baked yet.

https://github.com/kohya-ss/sd-scripts/commits/sd3/

I see. thanks for your fast response.

[bmaltais]

The current sd3 branch implement a MVP GUI over the se-scripts dev branch code… so it is likely not going to work perfectly just yet because the sd3 training code is not officially released. But it allows the community to test things out via the GUI as it progress… and possibly help as-scripts improve as you find bugs with it.

[bmaltais]

Well, I found that my code was not handling the text encoder parameter properly. I have fixed it. The training should work now.

[bmaltais]

OK... so I ran a test training a dreambooth and it does work. You will need a minimum of 24GB of VRAM. You can also look at my test config in the presets/dreambooth folder. Here is an example of the result:

Out of the box SD3:

Dreambooth SD3 trained for 5 minutes. Quite a transformation:

Prompt was: test4 woman standing by a window in a modern apartment, golden hour, photorealistic, award winning

[bmaltais]

Another example of the same model with a more complex prompt:

SD3:

Dreambooth:

Prompt: test4 woman wearing a pretty yellow dress sitting in a chair by a window in a modern apartment, a large flower vase by her side, golden hour, photorealistic, award winning

[GoofyAi]

Owner

cool. i do have 24gb vram. i was training a lora. do you have any preset for sd3 lora training?

[GoofyAi]

Well, I found that my code was not handling the text encoder parameter properly. I have fixed it. The training should work now.

after pulling the new changes the sd3 parameters are gone now. only the model sd3 selector is there

though i clicked the sd3 i get no sd3 exclusive parameters. the separate tab we had for sd3 parameters. when i hit the train button it executes the 1.5 trainer script

[bmaltais]

There is no sd3 Lora training support. Only dreambooth and fine tune yet. I will update the code to not show sd3 under Lora.

[GoofyAi]

There is no sd3 Lora training support. Only dreambooth and fine tune yet. I will update the code to not show sd3 under Lora.

not supported by the UI or the whole kohya script? because I saw one trainer can train lora. i hope to see lora support in your GUI though

[bmaltais]

Not supported by sd-scripts. You may want to stick to OneTrainer for Lora

[GoofyAi]

Not supported by sd-scripts. You may want to stick to OneTrainer for Lora

bummer. anyways any plan to add more model type training like hunyuan dit or pixart etc?

[bmaltais]

This is really a question for kohya. I only wrap his scripts with a GUI. So unless he add support for them I can't provide a GUI for it.

[BenDes21]

There is no sd3 Lora training support. Only dreambooth and fine tune yet. I will update the code to not show sd3 under Lora.

noob question but is it possible to finetune with SD3 then extract the lora from the file as sd3 lora ?

[bmaltais]

If Kohya_ss or someone else create a Lora extraction tool it will be possible. But to my knowledge it does not exist yet.

[GoofyAi]

If Kohya_ss or someone else create a Lora extraction tool it will be possible. But to my knowledge it does not exist yet.

do you think you can oficially support this? looks like they used your modified code to make the GUI https://github.com/Tencent/HunyuanDiT/tree/main/kohya_ss-hydit would be nice to see the official support

[bmaltais]

If Kohya_ss or someone else create a Lora extraction tool it will be possible. But to my knowledge it does not exist yet.

do you think you can oficially support this? looks like they used your modified code to make the GUI https://github.com/Tencent/HunyuanDiT/tree/main/kohya_ss-hydit would be nice to see the official support

Interesting. This is a fork that use a modified sd-scripts repo. I can't easilly integrate support for it. It is probably just best to use this seperate fork for that purpose. I might update the readme in my repo to point users looking for this type of finetuning to it.

[OliviaOliveiira]

Will you add support for diffusers format training? I reeeaally do not wish to redownload every single thing to start training + that'd save a lot of time rather than picking every single thing manually (i mean just point out stabilityai/stable-diffusion-3-medium-diffusers

[bmaltais]

You can try to pass the diffuser folder instead of the safetensor file as the model to fine tune. Ignore the clip models since they should be part of the diffuser folder.

if that work, cool. If not, you will need to ask kohya to add support for it directly on his as-scripts repository.

[GoofyAi]

If Kohya_ss or someone else create a Lora extraction tool it will be possible. But to my knowledge it does not exist yet.

do you think you can oficially support this? looks like they used your modified code to make the GUI https://github.com/Tencent/HunyuanDiT/tree/main/kohya_ss-hydit would be nice to see the official support

Interesting. This is a fork that use a modified sd-scripts repo. I can't easilly integrate support for it. It is probably just best to use this seperate fork for that purpose. I might update the readme in my repo to point users looking for this type of finetuning to it.

I found this repo from kblueleaf don't know if it can help you to support the dit model training in your ui as the modified ui is spitting alot of errors

https://github.com/KohakuBlueleaf/sd-scripts/tree/HunYuanDiT