search

StochasticAnalytics / emClarity

GNU Lesser General Public License v3.0
40 stars 6 forks source link

emClarity reconstruct requires an unreasonable amount of RAM #139

Open thomasfrosio opened 3 years ago

thomasfrosio commented 3 years ago

Hi,

The emClarity reconstruct program holds the entire dataset (+ temporaries?), on RAM, which ends up requiring too much RAM. For the EMPIAR-10304 dataset, containing about 10000 particles (190x190x190 Ang mask at bin1), emClarity reconstruct progressively increases to more than 400G of RAM.

Looking at the code, it looks like BH_to_cisTEM_mapBack is simply looping through each projection of each tilt-series and collects every particle tile. It first collects the tiles of a given projection on the GPU, in output_particle_stack: https://github.com/bHimes/emClarity/blob/f06204208e0617fc69e94696f8825d0077cdd637/synthetic/BH_to_cisTEM_mapBack.m#L527

Once the projection is done, the output stack is gathered to the RAM: https://github.com/bHimes/emClarity/blob/f06204208e0617fc69e94696f8825d0077cdd637/synthetic/BH_to_cisTEM_mapBack.m#L694

Ultimately, output_cell should contain the entire dataset...

Reducing the total exposure at the command line could be a temporary solution for the users (less exposure means less projections), but for large datasets I'm not sure emClarity reconstruct actually works.

Thomas

bHimes commented 9 months ago

TODO: change behavior to write stack out to disk incrementally.