Out of memory on CUDA

[Eronponce]

I already tried to reduce my batch to 1, and I am looking for help on how to run it better, my GPU has 6 GB of dedicated VRAM, and still is out of memory, maybe I need to reduce my sample?

[Jerry288]

Try resetting the runtime. Maybe that would work.

[jdakillah]

You can try to use google colab for generating the model, then afterwards you can copy the files to your server of Huggingface instance.

[remenraj]

Which model are you using? Maybe using a smaller model will help. Also lookup gradient accumulation, it can also help reduce the memory required for each batch.

[Spit-fires]

Use a lower batch size in args()