search

kohya-ss / sd-scripts

Apache License 2.0
4.54k stars 771 forks source link

Stable cascade stage b training #1260

Open riolys opened 3 months ago

riolys commented 3 months ago

When will stage b training code be released? Thanks!!

ashrafbay commented 3 months ago

Does c work at all?

riolys commented 3 months ago

Does c work at all?

It works. The code is good. However, the final decoding latent is produced by stage b. I think fine-tuning both would be better.

tristanwqy commented 3 months ago

Does c work at all?

It works. The code is good. However, the final decoding latent is produced by stage b. I think fine-tuning both would be better.

in the official repo, it says A quick clarification, Stable Cascade uses Stage A & B to compress images and Stage C is used for the text-conditional learning. Therefore, it makes sense to train a LoRA or ControlNet only for Stage C. You also don't train a LoRA or ControlNet for the Stable Diffusion VAE right?

riolys commented 3 months ago

Does c work at all?

It works. The code is good. However, the final decoding latent is produced by stage b. I think fine-tuning both would be better.

in the official repo, it says A quick clarification, Stable Cascade uses Stage A & B to compress images and Stage C is used for the text-conditional learning. Therefore, it makes sense to train a LoRA or ControlNet only for Stage C. You also don't train a LoRA or ControlNet for the Stable Diffusion VAE right?

That' right. But my training on stage c seems like the model has not learned the distribution of my dataset. I adjusted the learning rates from 1e-5 to 5e-6 but the results are similar. I think there are some issues I will consider: 1.The dataset quality. Although the data is high quality from the Internet and the labels are good, they are not from one unique distribution. I am trying another set. 2.The stage b does not know the data distribution. That way, stage b training is necessary.(stage a is good, I have tested it) I just tried the fine-tuning. I will try LoRA.

ashrafbay commented 3 months ago

Can you share the code please. I'm trying to train him using COLAB and KAGL but the errors are endless

riolys commented 3 months ago

I haven't completed the stage b training. maybe you can see here https://github.com/2kpr/StableCascade

ashrafbay commented 3 months ago

not work in cloab

kohya-ss commented 2 months ago

The training script for Stage B is more difficult than for C, and I'd like to prioritize other tasks. So I think it is better to use other repositories if they already work.

riolys commented 2 months ago

The training script for Stage B is more difficult than for C, and I'd like to prioritize other tasks. So I think it is better to use other repositories if they already work.

Thanks for reply. I find training stage b is not necessary. Stage C training is enough.