search

mshafieek / ADS-Missing-data-social-network

ADS master thesis
MIT License
0 stars 1 forks source link

Memory issue #16

Closed JesseJvdw closed 1 year ago

JesseJvdw commented 1 year ago

I found out that somewhere in my code something went wrong and I got the wrong estimates in both the true rem and the imputed list. Now I try to rerun my code but now I am running out of memory. My memory is able to run 74 simulations. Do you reckon that is enough or should I run 74 simulations a few time?

also getting a warning now: Warning: stack imbalance in 'cbind', 92 then 91

JesseJvdw commented 1 year ago

Just found out running it a few times also causes memory issues.

gerkovink commented 1 year ago

Try to save only the pooled model output. Have a look at the code that I provided for Vira (see the Gerko folder) and use a modular design that you can later stitch together. That helps with memory

gerkovink commented 1 year ago

What error do you receive exactly?

JesseJvdw commented 1 year ago

! 'Calloc' could not allocate memory (931680 of 16 bytes)

before this I had ! cannot allocate vector of size 14.2 Mb

gerkovink commented 1 year ago

Insufficient ram. Either increase the memory limit, if still possible, or run a modular simulation (e.g. 10 at the time) and then aggregate the nmodules x 10 pooled lists at the end. If you send or prepare for me a working RStudio project, I can try to run the one you have now for you tomorrow.

JesseJvdw commented 1 year ago

I'm trying to run a 1000 simulations but I'll check if I can do 100 with 10 at the time.

JesseJvdw commented 1 year ago

Managed to run a 100