Instancing a manager in multiprocessing module crashes graalpython

[xiaxinmeng]

In the module multiprocessing, if we create an object of "Manager", graalpython crashes.

test.py

from multiprocessing import  Manager

manager = Manager() #crash here
d = manager.dict({})

Error massage:

......
    _check_not_importing_main()
  File "/home/xxm/Desktop/IFuzzer/experiment_on_different_interpreter/graalpy/graalpy-22.3.1-linux-amd64/lib-python/3/multiprocessing/spawn.py", line 152, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
com.oracle.graal.python.runtime.exception.PythonExitException

Environment:

'graalpy-22.3.1-linux-amd64/bin/graalpy' Python 3.8.5 [Graal, GraalVM CE, Java 19.0.2] on Ubuntu 18.04

[msimacek]

It's the same cause as in #312, you need to wrap the code in if __name__ == '__main__':.

[xiaxinmeng]

I this this crash is slightly different from #312. After adding the statement "if name == 'main':" and running it with stream "with multiprocessing.Manager() as manager:"(see follow example), this case still produce a crash. And the crash information is different from #312 . The crash message is related to something about "active child contexts after finalizeContext" while #312 reports "Using setAllowActions is only permitted during finalization of a language."

import multiprocessing
if __name__ == '__main__':
    with multiprocessing.Manager() as manager:
        pass

Error message of this bug

[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[python::PythonContext] WARNING: Could not stop thread Polyglot-python-1
ERROR: java.lang.IllegalStateException: There is an active child contexts after finalizeContext!
org.graalvm.polyglot.PolyglotException: java.lang.IllegalStateException: There is an active child contexts after finalizeContext!
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.finishClose(PolyglotContextImpl.java:2481)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.closeImpl(PolyglotContextImpl.java:2387)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.closeAndMaybeWait(PolyglotContextImpl.java:1619)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.close(PolyglotContextImpl.java:1554)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextDispatch.close(PolyglotContextDispatch.java:73)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.close(Context.java:831)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.close(Context.java:857)
    at com.oracle.graal.python.shell.GraalPythonMain.launch(GraalPythonMain.java:681)
    at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:296)
    at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
    at org.graalvm.launcher.AbstractLanguageLauncher.runLauncher(AbstractLanguageLauncher.java:168)
Caused by: Attached Guest Language Frames (0)
Internal GraalVM error, please report at https://github.com/oracle/graal/issues/.

Error message of #312

ERROR: java.lang.IllegalStateException: Using setAllowActions is only permitted during finalization of a language. See TruffleLanguage.finalizeContext(Object) for further details.
org.graalvm.polyglot.PolyglotException: java.lang.IllegalStateException: Using setAllowActions is only permitted during finalization of a language. See TruffleLanguage.finalizeContext(Object) for further details.
    at org.graalvm.truffle/com.oracle.truffle.api.impl.ThreadLocalHandshake$TruffleSafepointImpl.setAllowActions(ThreadLocalHandshake.java:699)
    at com.oracle.truffle.llvm.runtime.LLVMContext.finalizeContext(LLVMContext.java:592)
    at com.oracle.truffle.llvm.runtime.LLVMLanguage.finalizeContext(LLVMLanguage.java:533)
    at com.oracle.truffle.llvm.runtime.LLVMLanguage.finalizeContext(LLVMLanguage.java:93)
    at org.graalvm.truffle/com.oracle.truffle.api.LanguageAccessor$LanguageImpl.finalizeContext(LanguageAccessor.java:326)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotLanguageContext.finalizeContext(PolyglotLanguageContext.java:404)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.finalizeContext(PolyglotContextImpl.java:2925)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.finishClose(PolyglotContextImpl.java:2473)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.closeImpl(PolyglotContextImpl.java:2387)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.closeAndMaybeWait(PolyglotContextImpl.java:1619)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.close(PolyglotContextImpl.java:1554)
    at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextDispatch.close(PolyglotContextDispatch.java:73)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.close(Context.java:831)
    at org.graalvm.sdk/org.graalvm.polyglot.Context.close(Context.java:857)
    at com.oracle.graal.python.shell.GraalPythonMain.launch(GraalPythonMain.java:681)
    at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:296)
    at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
    at org.graalvm.launcher.AbstractLanguageLauncher.runLauncher(AbstractLanguageLauncher.java:168)
Caused by: Attached Guest Language Frames (0)
Internal GraalVM error, please report at https://github.com/oracle/graal/issues/.

[msimacek]

You're right, I can reproduce it. I'll have a look.

[msimacek]

I did a large refactoring of the multiprocessing module to do true multiprocessing by default (it used to create isolated interpreter contexts in multiple threads of a single process). This bug should be fixed by that.