Open ericsnowcurrently opened 5 months ago
I think what's happening is:
module.__dict__
. This is not behavior that we want long term, but it's an important performance work around for 3.13. (See https://github.com/python/cpython/issues/117783)finalize_remove_modules
, but here that's not happening because the module's dictionary was immortalized.finalize_modules_clear_weaklist
when we start clearing the module's __dict__
in _PyModule_Clear
.By the time Interpreter.__del__
is called, a lot of global variables including _interpreters
have already been deleted from the module's dictionary so you get a bunch of weird errors, and the interpreter is not deleted.
That makes sense. You have any thoughts already on how to resolve this?
I'm not sure. We could call the finalizers on all GC-enabled objects during shutdown in the free-threaded build (before we start clearing module's dictionaries). I think that makes logical sense: everything is going away, why not call their finalizers? That seems to fix test_interpreters
, but a few other test cases would need tweaking.
Bug report
Bug description:
On one of my recent PRs, gh-118157, I was getting consistent failures on CI in test_interpreters. Different tests would fail, but the error was always the same:
(expand)
``` 0:08:24 load avg: 4.18 Re-running 1 failed tests in verbose mode in subprocesses 0:08:24 load avg: 4.18 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min) 0:08:26 load avg: 3.93 [1/1/1] test_interpreters failed (6 failures) Re-running test_interpreters in verbose mode (matching: test_display_preserved_exception, test_gh_109793, test_sys_path_0, test_sys_path_0, test_sys_path_0, test_sys_path_0) test_display_preserved_exception (test.test_interpreters.test_api.TestInterpreterExec.test_display_preserved_exception) ... FAIL test_gh_109793 (test.test_interpreters.test_lifecycle.FinalizationTests.test_gh_109793) ... FAIL test_sys_path_0 (test.test_interpreters.test_lifecycle.StartupTests.test_sys_path_0) ... ### start subtest debug ### # python3 script.py failed: --- stdout --- { "main": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4", "sub": "/tmp/test_python_tk7ha1xh/test_interpreters_76l2y2f4" } ------ --- stderr --- Exception ignored in:The error looks like this:
I'm fairly sure that means one of the 3 calls in
_interpreters._decref()
is raising a bogus exception.It isn't clear to me how gh-118157 would have caused this to happen. As a very temporary solution, I've skipped test_interpreters on free-threading builds.
CC @colesbury
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response