Open King4819 opened 6 months ago
Hi, you can use GradientCumulativeOptimizerHook
. Just set the dict
in the config file like this:
data = dict(samples_per_gpu=1)
optimizer_config = dict(type='GradientCumulativeOptimizerHook', cumulative_iters=2)
The total batch_size will be samples_per_gpu*cumulative_iters*num_gpus
.
@duanduanduanyuchen Thanks for your reply !
I want to ask that how to implement gradient accumulation on your work. Since my computing resource is single RTX4090 (24GB), so I'm not able to set batch size to 16, thanks !!!