search

ThirdAILabs / Demos

Notebooks for ThirdAI demos
Other
70 stars 13 forks source link

MemoryError: std::bad_alloc #110

Open Chirag-Mphasis opened 9 months ago

Chirag-Mphasis commented 9 months ago

Every time I try to run the generate.py script from the Git Repo it throws this error. The stack trace is pretty simple :

MemoryError Traceback (most recent call last) in <cell line: 8>() 6 MODEL_PATH = "/content/bolt2.5-generative.model" 7 tokenizer = GPT2Tokenizer.from_pretrained("gpt2") ----> 8 model = bolt.GenerativeModel.load(MODEL_PATH) 9 10

FYI, I am trying to run this on a Intel Sapphire rapids machine on GCP with 8 vCPU and 88GB memory. There is no way that it is running out of memory. I just wanted to know If I'm doing something fundamentally wrong

pratkpranav commented 9 months ago

Hi Chirag,

Thank you for bringing attention to this issue. It seems the error is happening during the model's de-serialization, potentially due to the latest updates in the thirdai package. Could you please try using the 0.7.20 release package by installing it via pip3 uninstall thirdai -y && pip3 install thirdai==0.7.20 and see if that solves the issue? In the meantime, we will focus on updating the model to be compatible with the newest updates in thirdai.