Using `embed` crashes with `ValueError: I/O operation on closed file`

[patrick-kidger]

Consider the following MWE:

❯ ptpython
>>> import ptpython
>>> try:
...     ptpython.embed()
... except SystemExit:
...     pass
>>> exit()  # from the nested interpreter

Here I am (a) using ptpython from the command line (as I really like it!) and (b) happen to calling .embed as part of my program; it has a try/except to recognise when the inner interpreter has terminated, to pass control back to the original invocation.

However, this produces an infinite loop of:

Traceback (most recent call last):
  File ".../site-packages/ptpython/repl.py", line 147, in run
    text = self.read()
           ^^^^^^^^^^^
  File ".../site-packages/ptpython/python_input.py", line 1100, in read
    result = self.app.run(pre_run=pre_run, in_thread=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/prompt_toolkit/application/application.py", line 954, in run
    raise exception
  File ".../site-packages/prompt_toolkit/application/application.py", line 939, in run_in_thread
    result = self.run(
             ^^^^^^^^^
  File ".../site-packages/prompt_toolkit/application/application.py", line 1002, in run
    return asyncio.run(coro)
           ^^^^^^^^^^^^^^^^^
  File ".../asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File ".../asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File ".../site-packages/prompt_toolkit/application/application.py", line 886, in run_async
    return await _run_async(f)
           ^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/prompt_toolkit/application/application.py", line 734, in _run_async
    with self.input.raw_mode(), self.input.attach(
         ^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/prompt_toolkit/input/vt100.py", line 123, in raw_mode
    return raw_mode(self.stdin.fileno())
                    ^^^^^^^^^^^^^^^^^^^
ValueError: I/O operation on closed file

Another way of framing this might be that ptpython does not evaluate programs in the same way as normal python: the above is a program with different behaviour.

[patrick-kidger]

Okay, I've figured this one out.

The issue is that builtins.{exit,quit} actually explicitly calls sys.stdin.close():

import dis
dis.dis(exit.__call__)

 19           0 RESUME                   0

 22           2 NOP

 23           4 LOAD_GLOBAL              0 (sys)
             16 LOAD_ATTR                1 (stdin)
             26 LOAD_METHOD              2 (close)
             48 PRECALL                  0
             52 CALL                     0
             62 POP_TOP
             64 JUMP_FORWARD             7 (to 80)
        >>   66 PUSH_EXC_INFO

 24          68 POP_TOP

 25          70 POP_EXCEPT
             72 JUMP_FORWARD             3 (to 80)
        >>   74 COPY                     3
             76 POP_EXCEPT
             78 RERAISE                  1

 26     >>   80 LOAD_GLOBAL              7 (NULL + SystemExit)
             92 LOAD_FAST                1 (code)
             94 PRECALL                  1
             98 CALL                     1
            108 RAISE_VARARGS            1

which seems a bit strange! (See also this related cpython discussion.)

This means it's actually possible to reproduce this issue in default python!

try:
    exit()
except SystemExit:
    pass
input()

The solution is to call embed with monkey-patched exit and quit methods:

globals = dict(globals())
globals["exit"] = sys.exit
globals["quit"] = sys.exit
ptpython.repl.embed(globals, ...)

(Note that sys.exit is different from builtins.exit.)