facebookresearch / multihop_dense_retrieval

Multi-hop dense retrieval for question answering
Other
214 stars 22 forks source link

OOM Error while training Electra QA model #23

Open todiketan opened 2 years ago

todiketan commented 2 years ago

Hello, thanks a lot for sharing the code for the paper. I was trying to train electra base model from scratch but the CPU RAM usage is increasing with every iteration, and eventually the process is getting killed due to the CPU RAM being full. The GPU RAM usage is constant across training. I am using a system with 64GB CPU RAM. Can any of the authors (or anyone who has trained or fine-tuned the QA model) share the exact version of pytorch used for the experiments, and did they face any similar issue while training the model?

Thanks in advance.