search

snap-stanford / GEARS

GEARS is a geometric deep learning model that predicts outcomes of novel multi-gene perturbations
MIT License
189 stars 38 forks source link

Any suggestions for the HPC requirement #35

Closed Marsrocky closed 10 months ago

Marsrocky commented 10 months ago

Really appreciate the delicate project and codes. I am running the code and find it stuck in the preprocessing phase due to the lack of RAM. My workstation is 128G RAM with 4 RTX 3090. Could you help suggest the minimum requirement for processing the data, e.g., norman dataset?

Thanks!

Marsrocky commented 10 months ago

@yhr91 The program is killed in "prepare_split()" with "simulation" mode. Thanks.

yhr91 commented 10 months ago

Hi, thanks for your question! We mainly used Nvidia 2080 Ti RTX 11GB GPUs to train GEARS. The node we used had 768GB of RAM.

Having said that, I think the RAM you have should be sufficient for training the model. I believe GEARS uses under 15GB of RAM during training on Norman. Are you sure you don't have other processes running in the background?

Marsrocky commented 10 months ago

Hi, thanks for your question! We mainly used Nvidia 2080 Ti RTX 11GB GPUs to train GEARS. The node we used had 768GB of RAM.

Having said that, I think the RAM you have should be sufficient for training the model. I believe GEARS uses under 15GB of RAM during training on Norman. Are you sure you don't have other processes running in the background?

Thanks for your prompt response. Yes I have checked the RAM usage of this one process, and it really took over 128G after a while. It stopped at the "prepare_split()" with the "simulation" mode. May I have the versions of the libraries in the requirements? This issue may arise from the library versions...

Marsrocky commented 10 months ago

Hi, thanks for your question! We mainly used Nvidia 2080 Ti RTX 11GB GPUs to train GEARS. The node we used had 768GB of RAM.

Having said that, I think the RAM you have should be sufficient for training the model. I believe GEARS uses under 15GB of RAM during training on Norman. Are you sure you don't have other processes running in the background?

The memory usage of more than 128G happens in the preprocessing period... Could you please help check if this also happens to your node? Many thanks!

Marsrocky commented 10 months ago

I have resolved this issue by upgrading the version of scanpy and pandas... Thanks for the help.