charmplusplus / charm4py

Parallel Programming with Python and Charm++
https://charm4py.readthedocs.io
Apache License 2.0
289 stars 21 forks source link

Clarify interaction with other parallel tools #176

Open RossBoylan opened 4 years ago

RossBoylan commented 4 years ago

The specific issue motivating this is that I'm thinking of spawning a process using either asyncio, subprocess or multiprocessing in the standard library.

  1. All those modules provides ways of waiting until the spawned process is complete. Will those work at all in charm4py? If they are in a method marked @coro, will they yield control, or just block until completion? https://charm4py.readthedocs.io/en/latest/rules.html says

    Note that coroutines are non-preemtive [typo] and only yield when they are waiting for something (typically occurs when they are blocked on a future or channel). When a method stops executing, control returns to the scheduler which will start or resume other methods on that process.

    The methods are waiting for something, but it is not any kind of a charm thing.

  2. asyncio has an event loop. Is that compatible with charm?

  3. multiprocessing warns against fork from a process with operating system threads. Does charm use such threads?

More generally, will the package interoperate properly with other things in base Python:

  1. Does basic file I/O qualify for the "waiting for something" condition of the documentation?
  2. What if you try to use threads?
  3. The newish native coroutine facility in Python along with await and related keywords?