Closed wbinventor closed 7 years ago
Currently, multi-node is limited to just the OpenMC side, but I have been mulling how to handle domain decomposition on the depletion operator.
For this particular problem, here is roughly where communication needs to happen:
With some modification to OpenMC, this can be made extremely parallelizable. Beyond this, I do not know what the plan is.
I was actually pointing to a seemingly much simpler issue. Let me rephrase - how would you submit a job on a cluster? I want to run OpenMC in parallel across nodes, but don't really care about distributed parallelism for OpenDeplete.
But of course the scheme you describe for running the depletion calculation in parallel makes sense to me. Do note for (1) that Pandas does indeed read the entire object from disk, but Dask and (PySpark)[http://spark.apache.org/docs/0.9.0/python-programming-guide.html] are designed for out-of-core applications.
Ah, in that case, just adjust the openmc call:
settings.openmc_call = ["mpiexec", "openmc"]
Depending on how it's set up, mpiexec should just link to your SLURM/PBS settings and not need be passed arguments. That is how I've been testing this on a supercomputer.
@cjosey alright I'll try that. I still think it would be nice to use the openmc.run
wrapper to more cleanly parametrize this.
Closed with #30.
Currently, OpenMC is simply run using
subprocess.call
at each timestep. It may be nice to instead use the more customizedopenmc.run(...)
method which has more flexible support for multi-processing/threading.In addition, some thought should be given to how one might deploy an OpenDeplete/MC job across multiple nodes. The need to be able to use 10s (or even 100s) of nodes is painfully obvious by simply trying to run a deplete a single 17x17 PWR fuel assembly, since OpenMC's runtime is inversely proportional to the number of nuclides in the problem.
@cjosey do you have any recommendations on how to use multiple nodes with your framework? Must something like
mpi4py
be incorporated to ensure that only a single Python process, perhaps MPI rank 0, is used to manage the simulation workflow?