What was the problem: I couldn't train the model on my laptop with 4GB of VRAM.
What I have done: I disabled gradient calculation during model validation (gradients were stored in VRAM). Now, training full model as is (USAD.ipynb) takes about 1.8 GB of VRAM.
What was the problem: I couldn't train the model on my laptop with 4GB of VRAM.
What I have done: I disabled gradient calculation during model validation (gradients were stored in VRAM). Now, training full model as is (USAD.ipynb) takes about 1.8 GB of VRAM.
This PR should fix #6 .