REPL: index out of range in REPL history

[picnixz]

Bug report

IMPORTANT

For anyone trying to reproduce this issue, please remove your Python history file before trying anything (remember to back it up) or use ./python -I instead so that you do not have surprises.

---

Bug description:

I don't really have a better title but here's what I wrote in the interpreter while trying to break metaclasses for this:

class A(type):
    def __prepare__(a, b): pass

class B(metaclass=A):
    pass

The inputs must be given line by line:

• write class A(type): ; press ENTER
• write def __prepare__(a, b): pass; press ENTER
• press ENTER
• write class B(metaclass=A): ; press ENTER
• write pass ; press ENTER
• press ENTER

The output is

>>> class A(type):
...     def __prepare__(a, b): pass
...
>>> class B(metaclass=A):
...     pass
...
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    class B(metaclass=A):
        pass
TypeError: A.__prepare__() must return a mapping, not NoneType

For now, this is fine. However, by hitting UP multiple times (5 times) until you cannot go up, you get:

>>> class B(metaclass=A):Traceback (most recent call last):
  File "/lib/python/cpython/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "/lib/python/cpython/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/lib/python/cpython/Lib/_pyrepl/__main__.py", line 51, in <module>
    interactive_console()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/lib/python/cpython/Lib/_pyrepl/__main__.py", line 48, in interactive_console
    return run_interactive(mainmodule)
  File "/lib/python/cpython/Lib/_pyrepl/simple_interact.py", line 154, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/lib/python/cpython/Lib/_pyrepl/readline.py", line 385, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/lib/python/cpython/Lib/_pyrepl/reader.py", line 768, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/lib/python/cpython/Lib/_pyrepl/reader.py", line 751, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/lib/python/cpython/Lib/_pyrepl/reader.py", line 690, in do_cmd
    command.do()
    ~~~~~~~~~~^^
  File "/lib/python/cpython/Lib/_pyrepl/commands.py", line 274, in do
    r.setpos_from_xy(x, new_y)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/lib/python/cpython/Lib/_pyrepl/reader.py", line 560, in setpos_from_xy
    if self.screeninfo[i][1][j] == 0:
       ~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Note that I've cleaned up my local history file before this and that I failed to get the IndexError (but still get a dirty history and bad up/down handling) with other inputs such as

>>> class A:
...     def foo(self): pass
...
>>> class B(metaclass=A):
...     pass
...

I suspect some newlines are badly handled but only if the error occurs at some point (I'm not sure if its tied to the fact that __prepare__ is bad, and whether I'm possibly leaving a module in a very bad state for some reason). I'm opening an issue because I'd like to know if someone is able to reproduce it or not. It might be my terminal emulator as well so I'd like confirmation.

The main commit is 2078eb45ca (I did not bisect the issue though, because it would take me a very long time since I don't know how to do it for interactive inputs like that).

Here's the hexdump of the python history:

00000000: 636c 6173 7320 4128 7479 7065 293a 0d0a  class A(type):..
00000010: 2020 2020 6465 6620 5f5f 7072 6570 6172      def __prepar
00000020: 655f 5f28 612c 2062 293a 2070 6173 730d  e__(a, b): pass.
00000030: 0a20 2020 200a 636c 6173 7320 4228 6d65  .    .class B(me
00000040: 7461 636c 6173 733d 4129 3a0d 0a20 2020  taclass=A):..
00000050: 2070 6173 730d 0a20 2020 200a             pass..    .

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[mdboom]

I can reproduce on a3711afefa7a, Debian/WSL/Windows Terminal.

[picnixz]

Good for me that I'm not alone in this then. I tried a bit more and it doesn't seem to be because of the metaclass handling. Actually, I also hit the same issue with this code:

class A:
    def foo(self):
        pass

class B(A):
    pass

This one is perfectly valid, although I still hit an IndexError while going up.

[pablogsal]

Can you try to see if reverting https://github.com/python/cpython/commit/32a0faba439b239d7b0c242c1e3cd2025c52b8cf helps?

[picnixz]

I confirm that reverting that commit helps. I also tried on main and the issue is still there, so reverting that specific commit fixed it!

[pablogsal]

CC @godlygeek

[pablogsal]

Can you check if https://github.com/python/cpython/issues/121531 fixes it?

[picnixz]

I can confirm that your PR fixes this specific issue (at least, I'm not able to reproduce what I described)

[pablogsal]

Fixed by https://github.com/python/cpython/pull/121531