python / cpython

The Python programming language
https://www.python.org/
Other
60.74k stars 29.32k forks source link

Flaky test `test_asyncio_repl_is_ok` in `test_repl` #119909

Open colesbury opened 3 weeks ago

colesbury commented 3 weeks ago

The test fails because the running the asyncio repl exits with:

Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads

I'm not sure if this is just a flaky test (i.e, _enter_buffered_busy failures are expected), or if it's a bug in the underlying repl.

The exit code -6 means the process crashes due to an abort(), I think.

Example: https://github.com/python/cpython/actions/runs/9331095550/job/25685390386?pr=119908

Test Output ``` ====================================================================== FAIL: test_asyncio_repl_is_ok (test.test_repl.TestInteractiveInterpreter.test_asyncio_repl_is_ok) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/test_repl.py", line 199, in test_asyncio_repl_is_ok assert_python_ok("-m", "asyncio") ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^ File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 180, in assert_python_ok return _assert_python(True, *args, **env_vars) File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 165, in _assert_python res.fail(cmd_line) ~~~~~~~~^^^^^^^^^^ File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/test/support/script_helper.py", line 75, in fail raise AssertionError("Process return code is %d\n" ...<13 lines>... err)) AssertionError: Process return code is -6 command line: ['/home/runner/work/cpython/cpython-builddir/hypovenv/bin/python', '-X', 'faulthandler', '-I', '-m', 'asyncio'] stdout: --- --- stderr: --- asyncio REPL 3.14.0a0 (remotes/pull/119908/merge-dirty:43e023a, Jun 1 2024, 14:47:23) [GCC 11.4.0] on linux Use "await" directly instead of "asyncio.run()". Type "help", "copyright", "credits" or "license" for more information. >>> import asyncio Exception in thread Interactive thread: Traceback (most recent call last): File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/asyncio/__main__.py", line 109, in run raise OSError(errno.ENOTTY, "tty required", "stdin") OSError: [Errno 25] tty required: 'stdin' During handling of the above exception, another exception occurred: exiting asyncio REPL... Traceback (most recent call last): Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name=''> at interpreter shutdown, possibly due to daemon threads Python runtime state: finalizing (tstate=0x000055910f31bbf0) Current thread 0x00007fa49bcdb740 (most recent call first): --- ---------------------------------------------------------------------- Ran 8 tests in 3.[354](https://github.com/python/cpython/actions/runs/9331095550/job/25685390386?pr=119908#step:22:355)s FAILED (failures=1) test test_repl failed ```
gvanrossum commented 3 weeks ago

Maybe @ambv or @pablogsal has a clue?

Eclips4 commented 3 weeks ago

It's look related to the 2237946af0981c46dc7d3886477e425ccfb37f28

ambv commented 3 weeks ago

The REPL fails to start because of stdin is not a tty:

 stderr:
---
asyncio REPL 3.14.0a0 (remotes/pull/119908/merge-dirty:43e023a, Jun  1 2024, 14:47:23) [GCC 11.4.0] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
Exception in thread Interactive thread:
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython-ro-srcdir/Lib/asyncio/__main__.py", line 109, in run
    raise OSError(errno.ENOTTY, "tty required", "stdin")
OSError: [Errno 25] tty required: 'stdin'

I'm fixing that case to revert to the classic REPL in this scenario.

ambv commented 3 weeks ago

I will need a better way to reproduce this. Inside asyncio/__main__.py the raised OSError in my code example is being handled by an except: block that indeed runs the old REPL in the case of a missing TTY. So there's nothing to fix there. I can force this path locally with nohup ./python.exe -m asyncio and it does what's expected.

So to see the exception bubble up as in the failing test, this must be a combination of this code running from a secondary daemon thread at interpreter shutdown.