Thanks for releasing your code. I replaced GPT2 with XLM-roberta-large, tensor dataset with LazyLoader, and did padding inside the data loader. Unfortunately, even with batch_size of 1, I got Cuda out of memory. Do you have any suggestions on how can I decrease memory usage? I have no problem with xlm-roberta-base.
Thanks for releasing your code. I replaced GPT2 with XLM-roberta-large, tensor dataset with LazyLoader, and did padding inside the data loader. Unfortunately, even with batch_size of 1, I got Cuda out of memory. Do you have any suggestions on how can I decrease memory usage? I have no problem with xlm-roberta-base.