out of memory on AWS ec2 p3.2xlarge, any ideas?

llSourcell / Doctor-Dignity

Doctor Dignity is an LLM that can pass the US Medical Licensing Exam. It works offline, it's cross-platform, & your health data stays private.

Apache License 2.0

3.83k stars 406 forks source link

out of memory on AWS ec2 p3.2xlarge, any ideas? #4

Open jmugan opened 11 months ago

jmugan commented 11 months ago

Hi, not a bug, but I tried to run this on AWS with 16G of memory on a Tesla V100-SXM2-16GB GPU and I couldn't load the model. It ran out of memory. Anyone know what might be going on? I ran it in colab just fine (at least loading the model) on 16GB memory. Thanks! .