superbit-collaboration / superbit-metacal

Contains a collection of routines used to perform gmix/metacalibration on simulated SuperBIT images
4 stars 1 forks source link

Implement batch parallelization using `multiprocessing` for mock generation #35

Closed sweverett closed 2 years ago

sweverett commented 2 years ago

As the title says. @mcclearyj having MPI issues on the Brown cluster (who doesn't), so this allows us to run using the builtin multiprocessing module. Still uses batch parallelization, and we can now allocate a bunch of cores on a node to each cluster or exposure without worrying about MPI.