training examples

[bghira]

we need internal training loop loss function, the same code used to do training time predictions

all i see is Lykon talking about how stable sd3.5L is but i dont see this bearing out in reality

think of my position as maintainer of simpletuner who wasnt really consulted at any point of the sd35 finetuning tutorial's creation.

the tutorial itself outlines the damage that occurred to the model when training. i find it hard to believe that this is normal results or mirrors in any way what was seen during pretraining.

it feels just like finetuning Flux.1 Schnell, but SD3L isnt even distilled, right?

anyway please provide the code examples needed or it feels like it is necessry for me to remove sd3.x support from simpletuner moving forward as it is not a good user experience to waste money.

[bghira]

learning rate 1e-5 for lycoris does nothing and then kaboom.. but flux trains fine.

[DriveHabits]

@bghira seems like a skill issue

[bghira]

if they provide the training example script then we can definitively state this one way or another. i would prefer it be a "skill issue".

[bghira]

well as a part of my contribution to the issue, i was using Flux VAE outputs by accident to train the model instead of SD3 VAE outputs. embarrassing, yes. but still, the lack of canonical training example leaves too much to the imagination when searching down issues and would still love to have access to one for reference.

updated test result;

[DarkViewAI]

well as a part of my contribution to the issue, i was using Flux VAE outputs by accident to train the model instead of SD3 VAE outputs. embarrassing, yes. but still, the lack of canonical training example leaves too much to the imagination when searching down issues and would still love to have access to one for reference.

updated test result;

as the guy said above, skill issue. P.s you seem very toxic and senile. You need to grow up bud, you constantly shitting on kohya and onetrainer is wild considering you've taken so much code from them.

do better.

[DarkViewAI]

love how you were shit talking lykon, but you were the one to make the mistake lol

[bghira]

i don't see your contributions anywhere..

[bghira]

this is from the official fine-tuning tutorial as published by StabilityAI

[bghira]

the tutorial is here; my trainer, simpletuner, was used for this tutorial because it gives the best results.

it gives the best results because i work hard on the implementations. i do deep dives, and investigate, and ask for help when i'm not seeing the expected results.

i want anyone to know that making mistakes and talking about them is fine, as it helps others who come along later with the same struggles.

accusing others of code theft or being "toxic" while simultaneously calling them senile is just kinda unwarranted, doesn't help push the discussion forward. if you think something has been stolen, try raising it as an issue on the bug tracker for either my project or kohya's or onetrainer, so that we can ensure proper attribution. however, it should be noted that i rely on Diffusers and have no need to steal code from anybody else. furthermore, simpletuner produces the best results and is updated to support new features on a more rapid pace than kohya's or nerogar's projects. those two are great developers, and i admire their tenacity and skill. i have no beef or issue with them, quite the opposite.

cc @brianfitzgerald @voletiv

[AmericanPresidentJimmyCarter]

Teaching machines to learn can make anyone grumpy, especially when you try to teach them nonsense and expect something else 😁