search

molecule-one / megan

Code for "Molecule Edit Graph Attention Network: Modeling Chemical Reactions as Sequences of Graph Edits"
MIT License
57 stars 19 forks source link

CUDA out of memory #9

Closed freehome1 closed 10 months ago

freehome1 commented 1 year ago

It's ok when I train on USPTO-50K and CUDA out of memory on USPTO-full image

freehome1 commented 1 year ago

It seems that the memory keep rising when training like this: image and finally oom

mikolajsacha commented 1 year ago

Hi, we trained all the models without any problems on a GPU with much smaller RAM (GeForce 1070), so I am not sure why this error occurs in your case. You can try using a GPU with more RAM (if you have such a possibility) or reducing the batch size. However, the batch size we use is already small (4), and reducing it could lead to unstable training.

Best, Mikołaj