Out of memory when running the example script on a single A40 GPU

kyegomez / Med-PaLM

Towards Generalist Biomedical AI

https://discord.gg/qUtxnK2NMf

MIT License

308 stars 46 forks source link

Out of memory when running the example script on a single A40 GPU #1

Closed AlonzoLeeeooo closed 1 year ago

AlonzoLeeeooo commented 1 year ago

Hi @kyegomez ,

Thanks for the amazing work! I was running the provided script on a single A40 GPU but it ran into a OOM error. Is Med-PaLM has to be implemented on A100 GPU? Or is there any solution for this OOM problem? Thanks you in advance for any reply.

Best,

kyegomez commented 1 year ago

@AlonzoLeeeooo Hey thank you for your appreciation😊 🙏 And, yes this happens alot, you 2 options:

Lower hyperparameters, just cut them in half from where they are now
Use smaller types like 16, 8bit

AlonzoLeeeooo commented 1 year ago

Thank you for the quick reply! I will try the provided solutions to see if it would work.

Best,