litebird / litebird_sim

Simulation tools for LiteBIRD
GNU General Public License v3.0
18 stars 13 forks source link

Map distribution over MPI processes #292

Open paganol opened 8 months ago

paganol commented 8 months ago

It would be nice to have a function that reads maps (or generates maps with MBS) and spreads it over all the MPI processes.

mreineck commented 8 months ago

Are you thinking about an approach like this (pseudocode!)?

def generate_and_broadcast(generator, comm):
    result = generator() if comm.rank == 0 else None
    return comm.bcast(result)

generator can be a lambda or a function that either loads maps from disk or computes them on the fly.

paganol commented 8 months ago

Hi @mreineck, as first implementation, yes I was thinking to something like that. Than, probably, we should write something more refined for broadcasting properly maps, alms or other quantities based on the detector distribution across the mpi tasks.

paganol commented 3 weeks ago

I think this is addressed by #334, right?

ziotom78 commented 3 weeks ago

I think this is addressed by #334, right?

334 addresses the distribution of detectors, so if MBS generates maps according to which detectors are available on an MPI node, then yes, that PR addresses this issue.