New REPL will exit the interpreter due to errors from substituting builtins.__import__

[devdanzin]

Bug report

Bug description:

It's possible to make the new REPL exit the interpreter by replacing builtins.__import__ with a function that takes the wrong number of arguments or returns an invalid object. The basic REPL still works in these cases, even if imports don't. This happens in Linux and Windows with main.

>>> import builtins
>>> builtins.__import__ = lambda: None
>>> Exception ignored in the internal traceback machinery:
TypeError: <lambda>() takes 0 positional arguments but 5 were given
Traceback (most recent call last):
  File "/home/danzin/projects/cpython/Lib/runpy.py", line 198, in _run_module_as_main
  File "/home/danzin/projects/cpython/Lib/runpy.py", line 88, in _run_code
  File "/home/danzin/projects/cpython/Lib/_pyrepl/__main__.py", line 6, in <module>
  File "/home/danzin/projects/cpython/Lib/_pyrepl/main.py", line 59, in interactive_console
  File "/home/danzin/projects/cpython/Lib/_pyrepl/simple_interact.py", line 147, in run_multiline_interactive_console
  File "/home/danzin/projects/cpython/Lib/_pyrepl/readline.py", line 385, in multiline_input
  File "/home/danzin/projects/cpython/Lib/_pyrepl/reader.py", line 773, in readline
  File "/home/danzin/projects/cpython/Lib/_pyrepl/reader.py", line 722, in handle1
  File "/home/danzin/projects/cpython/Lib/_pyrepl/unix_console.py", line 549, in input_hook
TypeError: <lambda>() takes 0 positional arguments but 5 were given

Python 3.14.0a0 (heads/main:9187484dd97, Jul 29 2024, 09:25:22) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import builtins
>>> builtins.__import__ = lambda x, y, z, a, b: None
>>> Exception ignored in the internal traceback machinery:
Traceback (most recent call last):
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 139, in _print_exception_bltin
    return print_exception(exc, limit=BUILTIN_EXCEPTION_LIMIT, file=file, colorize=colorize)
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 129, in print_exception
    te = TracebackException(type(value), value, tb, limit=limit, compact=True)
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 1045, in __init__
    self.stack = StackSummary._extract_from_extended_frame_gen(
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 492, in _extract_from_extended_frame_gen
    f.line
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 369, in line
    self._set_lines()
  File "/home/danzin/projects/cpython/Lib/traceback.py", line 350, in _set_lines
    lines.append(linecache.getline(self.filename, lineno).rstrip())
  File "/home/danzin/projects/cpython/Lib/linecache.py", line 25, in getline
    lines = getlines(filename, module_globals)
  File "/home/danzin/projects/cpython/Lib/linecache.py", line 41, in getlines
    return updatecache(filename, module_globals)
  File "/home/danzin/projects/cpython/Lib/linecache.py", line 100, in updatecache
    stat = os.stat(fullname)
AttributeError: 'NoneType' object has no attribute 'stat'
Traceback (most recent call last):
  File "/home/danzin/projects/cpython/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/danzin/projects/cpython/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "/home/danzin/projects/cpython/Lib/_pyrepl/__main__.py", line 6, in <module>
    __pyrepl_interactive_console()
  File "/home/danzin/projects/cpython/Lib/_pyrepl/main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
  File "/home/danzin/projects/cpython/Lib/_pyrepl/simple_interact.py", line 147, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/home/danzin/projects/cpython/Lib/_pyrepl/readline.py", line 385, in multiline_input
    return reader.readline()
  File "/home/danzin/projects/cpython/Lib/_pyrepl/reader.py", line 773, in readline
    self.handle1()
  File "/home/danzin/projects/cpython/Lib/_pyrepl/reader.py", line 722, in handle1
    input_hook = self.console.input_hook
  File "/home/danzin/projects/cpython/Lib/_pyrepl/unix_console.py", line 552, in input_hook
    if posix._is_inputhook_installed():
AttributeError: 'NoneType' object has no attribute '_is_inputhook_installed'

Not sure this is something that should be guarded against or is just a "then don't do that" case.

If this is deemed worth fixing, we could change _pyrepl.windows_console.[Windows|Unix]Console.input_hook to guard against TypeError and AttributeError, or move the imports to module level. Also it might make sense to guard updatecache in linecache against the AttributeError. But, again, I'm not sure this should be fixed.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux, Windows

[terryjreedy]

I think touching builtins is 'at your own risk' territory. I am a little surprised that the repl is apparently trying to execute code after printing the prompt and before anything is entered. Did you hit return to enter a blank line? Or was an entered line overwritten by the exception? (The latter might be considered a bug.)

In 3.12.4, a subsequent import itertools, for instance, is required to get a simple traceback with the same 'takes 0' exception. It is followed by a new prompt.

When I do the same in IDLE's Shell, which executes entered code in a subprocess, I get this unusual output on stdout:

----------------------------------------
Unhandled exception in user code execution server!'
Thread: SockThread
IDLE Client Address: ('127.0.0.1', 49477)
Request: <socket.socket fd=572, family=2, type=1, proto=0, laddr=('127.0.0.1', 60874), raddr=('127.0.0.1', 49477)>

followed by a traceback on stderr from the socket thread of the subproccess, ending with the 'take 0' exception, followed by another traceback from the subprocess exception handler, again with the 'take 0' exception. I do not consider this a bug in IDLE. Replace a support girder with a defective version and the structure collapses? The user's fault.

[devdanzin]

I think touching builtins is 'at your own risk' territory. I am a little surprised that the repl is apparently trying to execute code after printing the prompt and before anything is entered. Did you hit return to enter a blank line? Or was an entered line overwritten by the exception? (The latter might be considered a bug.)

No, I didn't hit return: the issue comes from e.g. WindowsConsole.input_hook, which is a property: https://github.com/python/cpython/blob/46f5a4f9e1781ad8d60eb53bbaf6cd8534a286cd/Lib/_pyrepl/windows_console.py#L204-L211

That is checked at the loop in _pyrepl.reader.Reader.handle1: https://github.com/python/cpython/blob/46f5a4f9e1781ad8d60eb53bbaf6cd8534a286cd/Lib/_pyrepl/reader.py#L726-L737

Here's a diff that fixes both errors in Windows and avoids exiting the interpreter:

diff --git a/Lib/_pyrepl/windows_console.py b/Lib/_pyrepl/windows_console.py
index ba9af36b8be..0d98e4d5da0 100644
--- a/Lib/_pyrepl/windows_console.py
+++ b/Lib/_pyrepl/windows_console.py
@@ -205,10 +205,11 @@ def refresh(self, screen: list[str], c_xy: tuple[int, int]) -> None:
     def input_hook(self):
         try:
             import nt
-        except ImportError:
+            if nt._is_inputhook_installed():
+                return nt._inputhook
+        except (ImportError, AttributeError, TypeError):
             return None
-        if nt._is_inputhook_installed():
-            return nt._inputhook
+

     def __write_changed_line(
         self, y: int, oldline: str, newline: str, px_coord: int
diff --git a/Lib/traceback.py b/Lib/traceback.py
index 6ee1a50ca68..744d9605238 100644
--- a/Lib/traceback.py
+++ b/Lib/traceback.py
@@ -695,7 +695,7 @@ def output_line(lineno):
         return ''.join(row)

     def _should_show_carets(self, start_offset, end_offset, all_lines, anchors):
-        with suppress(SyntaxError, ImportError):
+        with suppress(SyntaxError, ImportError, TypeError):
             import ast
             tree = ast.parse('\n'.join(all_lines))
             statement = tree.body[0]