GPU memory rises continuously

salesforce / ALBEF

Code for ALBEF: a new vision-language pre-training method

BSD 3-Clause "New" or "Revised" License

1.53k stars 195 forks source link

GPU memory rises continuously #66

Open yuanrr opened 2 years ago

yuanrr commented 2 years ago

When I finetune the ALBEF on VQA task, I find that the GPU memory rises continuously. It ends up with 1.5 times as much GPU memory as it started with. Is this situation normal? How could I solve this problem.....THANKS

LiJunnan1992 commented 2 years ago

Can I know who much GPU memory it occupies? A memory rise at the beginning of training should be normal.