search

AI4HealthUOL / SSSD

Repository for the paper: 'Diffusion-based Time Series Imputation and Forecasting with Structured State Space Models'
MIT License
270 stars 47 forks source link

CUDA out of memory issue #24

Closed chomgid closed 3 months ago

chomgid commented 4 months ago

Hi I'm currently trying to replicate experiments from the paper using your code and NVIDIA RTX A5000(24G memory capacity), but getting memory issues. To be more explicit, I'm using PTB-XL dataset identical to that from the paper. Did you run the code on NVIDIA A30 card(which also has 24G memory capacity) as presented in the paper? If so, could there be any other reason why I'm getting these errors?

Thanks for reading :)

chomgid commented 4 months ago

Oh and I had to reduce the size of residual layers from 36 to 12 in the end to make the code runnable. I'm just wondering if you have gone through similar problems.

juanlopezcode commented 3 months ago

Hi @chomgid, I used an NVIDIA A30 card (24GB memory) for the experiments. If you're facing higher memory usage, it might be due to multiple processes simultaneously, e.g. other files with stuff loaded into the card, usually, the model + ptbxl dataset takes almost 24gb. I hope you were able to resolve your issue. best, Juan.