Can i input the parameter "gradient accumulation steps" in some way?

[RaySteve312]

Can i input the parameter "gradient accumulation steps" in some way? I watched several training tutorials, they all mentioned this parameter."Gradient accumulation steps" can be used with batch size. batch size * gradient accumulation steps= X X usually means the better quality of output model, but high batch size also costs more vram than most people can afford. Here "Gradient accumulation steps" can help people use more training time instead of more vram to achieve the same output quality as higher batch size does.

for example, most people can't afford to use batch size 20 gradient accumulation steps 1(which is 20) to train, but they can afford to use batch size 4 gradient accumulation steps 5 to achieve the same result.

[kijai]

You can input it in the additional_args box:

[Orenji-Tangerine]

You can input it in the additional_args box:

Hi Kijai, I notice your node has train_text_encoder, may I know it is for what purpose? Is that for the next update?

[kijai]

You can input it in the additional_args box:

Hi Kijai, I notice your node has train_text_encoder, may I know it is for what purpose? Is that for the next update?

It's currently under testing in another branch, it allows training a LoRA for the T5 as well.

[RaySteve312]

thanks for your work! you are my hero!

[RaySteve312]

You can input it in the additional_args box:

I want to input 2 args at the same time. but it doesn't work. When I input either arg of them, it works. Is the input format wrong?

[kijai]

You can input it in the additional_args box:

I want to input 2 args at the same time. but it doesn't work. When I input either arg of them, it works. Is the input format wrong?

Yeah I think I had an error there, it's fixed in my current train_t5 branch which I can hopefully merge soon. It will be parsed like command line arguments would.

[RaySteve312]

You can input it in the additional_args box:

I want to input 2 args at the same time. but it doesn't work. When I input either arg of them, it works. Is the input format wrong?

Yeah I think I had an error there, it's fixed in my current train_t5 branch which I can hopefully merge soon. It will be parsed like command line arguments would.

that would be great!🍻

[RaySteve312]

You can input it in the additional_args box:

I want to input 2 args at the same time. but it doesn't work. When I input either arg of them, it works. Is the input format wrong?

Yeah I think I had an error there, it's fixed in my current train_t5 branch which I can hopefully merge soon. It will be parsed like command line arguments would.

one more question, how to input reg images and repeat reg images, and set a weight? I tried to use the command line from https://github.com/kohya-ss/sd-scripts/blob/main/docs/config_README-en.md, but seems it didn't work.

since regulariation plays an important role in training styles lora, I just read some introduction about it, and I find L2 in your node Init Flux Training. It's soooo much to learn.