search

Tianxiaomo / pytorch-YOLOv4

PyTorch ,ONNX and TensorRT implementation of YOLOv4
Apache License 2.0
4.47k stars 1.49k forks source link

Hi, I want to know what's the meaning of subdivisions #463

Closed GuoQuanhao closed 3 years ago

GuoQuanhao commented 3 years ago 
if global_step % config.subdivisions == 0:
    optimizer.step()
    scheduler.step()
    model.zero_grad()
lixinghe1999 commented 3 years ago

From my understanding, it accumulate multiple times of gradient descent, so when your GPU can only afford 4 batchsize, you can simulate that it can afford 16(4*4). Of course, this trick will not make your training faster, it only may help it converge.

GuoQuanhao commented 3 years ago

Oh, thanks, I think you are right. I think it's amazing, I have never seen this.