Allow multiple python interpreters per process

[dbnicholson]

Checklist

• [x] the issue is indeed a bug and not a support request
• [x] issue doesn't already exist: https://github.com/kivy/python-for-android/issues
• [ ] I have a short, runnable example that reproduces the issue
• [x] I reproduced the problem with the latest development version (p4a.branch = develop)
• [x] I used the grave accent (aka backticks) to format code or logs when appropriated

Versions

• Python: 3.9
• OS: Linux
• Kivy: N/A
• Cython: N/A
• OpenJDK: 11

Description

Currently the native code starting the python interpreter only handles a single interpreter per process. This limits how applications handle services (and soon workers) as the python interpreter always has to be run in a separate process instead of a separate thread in the current process. This is managed in the AndroidManifest.xml file by declaring android:process in the service declarations.

Python does support this via sub-interpreters, but the setup is more complex. It would be nice to support that, though. Particularly for workers where they currently have to use more complex RemoteListenableWorker class.

[rtibbles]

@dbnicholson and I have talked about this separately, but just to add in from my experience of attempting to handle this, I think that it wouldn't even be necessary to support sub-interpreters (which is apparently somewhat finicky https://docs.python.org/3/c-api/init.html#bugs-and-caveats), but just properly register the non-Python created threads against the GIL.

However, in pursuing these approaches, I have come across an issue that seemingly is specific to PyJnius, whereby when it attempts to import its interface to Java from the pyi files, it results in an import not found error. The only possible interpretation that I have been able to come up with so far is that somehow the imported modules are garbage collected, because all Python threads have stopped, and this somehow does not get remedied when reinitializing Python in a subsequent invocation. I will report more here as I uncover it.