Open grovessss opened 2 months ago
sure thank you! can I ask why simply setting gradient_accumulation_steps=args.gradient_accumulation_steps in accelerator definition doesn't work?
you will need something like
thank you very much!
maybe it's not so relevant with your method, but i was wondering with gradient accumulation, is there anything special that needed to be done regarding the optimization at different frequencies process in your method? or can accelerate do it all
we don't have the code for grad accumulation. You can potentially modify https://github.com/tianweiy/DMD2/blob/main/main/train_sd.py to achieve this though