search

kohya-ss / sd-scripts

Apache License 2.0
4.79k stars 804 forks source link

[SDXL] Train SDXL refiner for subject generation #620

Open jkcarney opened 1 year ago

jkcarney commented 1 year ago

I've been using the scripts here to fine tune the base SDXL model for subject driven generation to good effect. However, I've found that adding the refiner step usually means that the refiner doesn't understand the subject, which often makes using the refiner worse with subject generation.

Currently the scripts do not support fine tuning the refiner to make generation better. When tuning the refiner, realistically we would only need to fine tune the text encoder, not the u-net. If this ability was added, I think that would be very beneficial.

TingTingin commented 1 year ago

Yes this would be great

wwirving commented 1 year ago

+1 for this!

ymzlygw commented 1 year ago

Agree! Training Refiner model is really important now if community want to get a good anime model like NAI. They must train a anime refiner, only base fine-tune is not enough.

singularity24601 commented 11 months ago

+1 to this. SDXL base 1.0 is good, but the refiner is what elevates it to great.

yamkz commented 10 months ago

+1 to this

FeiArt-Ai commented 10 months ago

I want this too

MichaelFan01 commented 9 months ago

I want this too

mio-nyan commented 9 months ago

Are there any docs or papers about the refiner? Something to start with?