search

bghira / SimpleTuner

A general fine-tuning kit geared toward diffusion models.
GNU Affero General Public License v3.0
1.49k stars 131 forks source link

[sdxl refiner] allow manual import of aesthetic scores #398

Closed bghira closed 2 weeks ago

bghira commented 3 months ago

a user might want to manually score their dataset and then use those values instead of the single faked value.

otoh it can also be useful to import a score list from a script that pre-processes the dataset as well.

but currently, the single faked value just seems useful for eg. improving the refiner at a specific value rather than a large finetune across the board.

sneccc commented 3 months ago

a bit unrelated but is it possible to use score models while training to guide finetune to more aesthetic pleasing results? i have seen something similar in Onetrainer with align prop https://arxiv.org/abs/2310.03739 but results are questionable afaik no one did a deep dive into it

Possibly this could speed up convergence since its filtering the ugly stuff, artifacts etc