search

tomassosorio / OCR_tablenet

TableNet Implementation on Pytorch
144 stars 45 forks source link

Gpu getting filled up after 3-4 epoch #15

Closed njnitesh82 closed 3 years ago

njnitesh82 commented 3 years ago

Hi, I wrote the tablenet code with the resnet backbone with the help of your code, but there one issue which I was not able to solve that. while the training the GPU memory get filled up after 3-4 epoch. Please help me.

I am giving a link to my model code. https://discuss.pytorch.org/t/memory-getting-filled-up-after-3-or-4-epoch/118333?u=njnitesh

tomassosorio commented 3 years ago

Hi @njnitesh82 , could you also provide a snipped of your training module? Are you running optimizer.zero_grad() at the beginning of each iteration? Otherwise, you might be accumulating the calculated gradients. Since I do not know the size of your dataset or compute instance this is my best guess.