hpc4cmb / toast

Time Ordered Astrophysics Scalable Tools
Other
44 stars 39 forks source link

Expected memory usage in TOAST satellite sim #199

Closed zonca closed 6 years ago

zonca commented 6 years ago

In the PICO noise simulations, I am running with just 1 pixel and 1 MPI process per node and I can fit 3 days worth of data per node. I am using a configuration very similar to the tiny satellite example for everything else. PICO is 1 KHz, so I expect 1 day of 1 pixel to occupy 1000*3600*24*8*2/1e9 = 1.4 GB

so 3 days would be 4.2 GB

I see madam allocating 6.1 GB for TOD + 14 GB pointing.

TiMemory reports 34 GB current and 53 GB peak.

I am just checking if you think all these numbers look reasonable, so I can use the metric of fitting between 4 GB and 5 GB of data per node to guess the initial size of jobs.

zonca commented 6 years ago

yes! it ran to completion, using a max of 7.7GB ram per MPI process

tskisner commented 6 years ago

Closed by #236