Support SPMD initialization of a multi-locale / global-view Chapel program

[bradcray]

Imagine the following scenario:

• I'm running an MPI-based SPMD programming model that owns main() and was launched by mpirun, called MPI_Init(), etc.
• I want to create a Chapel library that can be invoked by that program, yet operate in a multi-locale mode (global namespace, array of locales, on-clauses, etc.)
• specifically, I want to have each process in my SPMD programming model act like a locale in Chapel and support the ability to communicate, use the global address space, etc.
• I'd like to have the Chapel library operate in a CHPL_COMM=gasnet, CHPL_COMM_SUBSTRATE=mpi mode

What would our runtime need to expose (if it doesn't already) in order to support this style of running using the local processes? In a quick discussion with the GASNet team on Slack, it sounds as though GASNet's gex_Client_Init() should work in this mode and be able to use the existing processes, so the main questions are probably in the Chapel runtime itself.

[bradcray]

Refreshing my memory of the runtime's bootstrapping procedure, it looks like the question would be whether chpl_rt_init() could simply be called, or whether we'd need to refactor it to avoid some steps and make sure to take care of some others. Since we explicitly parse the number of locales out of the argc/argv arguments passed to it, presumably we'd either need to send in the equivalent of -nl # (with the appropriate # filled in) or refactor it to avoid argc/argv parsing and just set the number of locales directly.

[lydia-duncan]

Library compilation creates a different entrance point than executable compilation, chpl_library_init. It calls chpl_rt_init under the covers (as does main for executables). If it makes sense to share a significant portion of the code then I can see altering chpl_rt_init being the right way to go, but we could also consider splitting the functionality further if there wasn't much overlap

[bradcray]

Thanks for that reminder Lydia — Just to make sure I'm not completely off-base here, I was thinking that existing Chapel libraries either only supported single-locale execution (e.g., I'd create a library that could be called from one node or from multiple nodes using it as a local library only, but not in a way that supported global-view, multi-locale interactions); or multi-locale execution using a client-server model, as with the Python multi-locale library cases. Have I got that right?

[lydia-duncan]

Yes, though there's been a longstanding desire to investigate what it would take to get multi-locale library compilation to be more amenable to running in a single server/multiple client set up (or at least more similar to how Arkouda operates today, if I reversed things). I could see providing information about how to set up the locale situation as probably beneficial to both that use case and this one