Support CPython 3.8 on macOS: multiprocessing does not fork anymore

From https://docs.python.org/3/library/multiprocessing.html:

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

On macOS, gipc so far relied on multiprocessing to use the fork()-based method, just as it does on Linux. Now that this changed many assumptions are violated resulting in many tests to fail. For example:

def test_handlecount(self):
        p = start_process(ipc_handlecounter1, args=(self.rh, self.rh2))
        p.join()
>       assert p.exitcode == 0
E       assert 1 == 0
E         -1
E         +0
test/test_gipc.py:417: AssertionError
----------------------------- Captured stderr call -----------------------------
12:48:24,969.2  [17769]start_process# Invoke target `<function ipc_handlecounter1 at 0x106842af0>` in child process.
12:48:24,976.1  [17769]start# SIGCHLD watcher for 17831 started.
12:48:24,977.3  [17769]start_process# Invalidate <_GIPCReader_3a65e2 fd: 9> in parent.
12:48:24,977.6  [17769]close# Invalidating <_GIPCReader_3a65e2 fd: 9> ...
12:48:24,977.8  [17769]start_process# Invalidate <_GIPCReader_ac0ac9 fd: 15> in parent.
12:48:24,977.9  [17769]close# Invalidating <_GIPCReader_ac0ac9 fd: 15> ...
12:48:25,370.8  [17831]_child# _child start. target: `<function ipc_handlecounter1 at 0x108dfe790>`
12:48:25,372.4  [17831]_child# Delete current hub's threadpool.
12:48:25,372.8  [17831]_child# Destroy hub and default loop.
12:48:25,373.1  [17831]_child# Created new hub and default event loop.
12:48:25,373.2  [17831]_set_legit_process# Legitimate <_GIPCReader_3a65e2 fd: 9> for current process.
12:48:25,373.3  [17831]_child# Handle `<_GIPCReader_3a65e2 fd: 9>` is now valid in child.
12:48:25,373.4  [17831]_set_legit_process# Legitimate <_GIPCReader_ac0ac9 fd: 15> for current process.
12:48:25,373.5  [17831]_child# Handle `<_GIPCReader_ac0ac9 fd: 15>` is now valid in child.
Process _GProcess-14:
Traceback (most recent call last):
  File "/Users/travis/.travis-pyenv/versions/3.8.3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/Users/travis/.travis-pyenv/versions/3.8.3/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/travis/.travis-pyenv/versions/3.8.3/envs/ve-pyenv-3.8.3/lib/python3.8/site-packages/gipc/gipc.py", line 375, in _child
    target(*args, **kwargs)
  File "/Users/travis/build/jgehrcke/gipc/test/test_gipc.py", line 558, in ipc_handlecounter1
    assert len(get_all_handles()) == 2
AssertionError
12:48:25,410.5  [17769]_on_sigchld# SIGCHLD watcher callback for 17831 invoked. Exitcode stored: 1
------------------------------ Captured log call -------------------------------
DEBUG    gipc:gipc.py:285 Invoke target `<function ipc_handlecounter1 at 0x106842af0>` in child process.
DEBUG    gipc:gipc.py:509 SIGCHLD watcher for 17831 started.
DEBUG    gipc:gipc.py:303 Invalidate <_GIPCReader_3a65e2 fd: 9> in parent.
DEBUG    gipc:gipc.py:657 Invalidating <_GIPCReader_3a65e2 fd: 9> ...
DEBUG    gipc:gipc.py:303 Invalidate <_GIPCReader_ac0ac9 fd: 15> in parent.
DEBUG    gipc:gipc.py:657 Invalidating <_GIPCReader_ac0ac9 fd: 15> ...
DEBUG    gipc:gipc.py:523 SIGCHLD watcher callback for 17831 invoked. Exitcode stored: 1

https://travis-ci.org/github/jgehrcke/gipc/jobs/688793200#L11121

The action item here is adjust gipc to the new multiprocessing behavior of CPython 3.8 on macOS. A non-trivial effort.

jgehrcke / gipc

Support CPython 3.8 on macOS: multiprocessing does not fork anymore #100