search

nuclear-multimessenger-astronomy / nmma

A pythonic library for probing nuclear physics and cosmology with multimessenger analysis
https://nuclear-multimessenger-astronomy.github.io/nmma/
GNU General Public License v3.0
33 stars 58 forks source link

Zenodo when running in mpiexec with multiple cores #126

Closed mcoughlin closed 1 year ago

mcoughlin commented 1 year ago

On large clusters, it is common to what to run mpiexec -np 18 light_curve_analysis ... which attempts to trigger the Zenodo download 18 times, rather than start it once in one thread and block until completion. It would be nice to add some code where one thread starts the download and assigns some kind of temporary file that the other threads wait til are removed before continuing onwards.

Theodlz commented 1 year ago

Hi @mcoughlin, not possible to just trigger the download separately (I can make it so the utils/models.py has an argparse) before running that command? That seems very reasonable.

mcoughlin commented 1 year ago

@Theodlz Yes that would be fine. Some example command line with just the model name and svdpath or similar which just downloads sounds like a good way.

mcoughlin commented 1 year ago

Closing assuming that the download script is sufficient for this.