How to avoid the issue of CUDA out of memory?

calubkk / RAAT

[ACL-2024]Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training

13 stars 1 forks source link

How to avoid the issue of CUDA out of memory? #3

Closed sunxin000 closed 1 month ago

sunxin000 commented 1 month ago

Hello,

I found your paper very interesting. However, when I run the code, I encounter the "CUDA out of memory" error, even with four 80G A100 GPUs. Do I need to implement mixed precision operations? Could you please provide your accelerate configuration? That would be very helpful. Thank you so much!

calubkk commented 1 month ago

Thank you for your interest in our work. Here is the configuration. You can also use ‘accelerate config’ command to implement deepspeed.