Open jmacedoit opened 1 year ago
Hi @jmacedoit,
First of all, I am extremely sorry about this late reply. I have no idea why I didn't get any notification. I hope you already figured it out somehow.
The training command is at the bottom of the readme file. If the library versions ("pytorch-lightning==1.5.10", "transformers>=4.16.2") are the same as in the setup.py file and these arguments (--device gpu --gpus -1 --strategy ddp) are passed in the command line, there shouldn't be any problem. Also, batch size and the number of negative examples ratio might need to be adjusted for fitting in GPU memory.
Please let me know if you already solved it or if this helps.
Best wishes, sayef
Greetings
First of all I would like to thank you for this library.
I was trying to perform a train but ran out of VRAM as soon as the first epoch started. I'm using 2xA100 40Gb. Do I still need more GPUs? Through
nvidia-smi
I can attest that both GPUs were being used.Also, I'm using
--pretrained-model bert-base-multilingual-uncased
.