Open nfnfgo opened 1 day ago
I'm on Windows 10 and I've tested with the built-in Powershell (5.1.19041.5007) and I installed Powershell 7 (7.4.5). Unfortunately, I can't reproduce the behavior you're seeing.
Can you run pip freeze
to check what version of pyreadline3 is installed? I am using 3.5.4, which is the newest version.
Thanks for the reply!
I was using pyreadline3==3.4.1
while recording the demo. However, the issue still persists even after upgrading to pyreadline3==3.5.4
.
I'm managing my environment with conda
, but I'm not sure if the problem is related to using conda in the terminal.
I also tried uninstalling and reinstalling the latest version of cmd2
, but that didn’t fix it.
You mentioned you're using Windows 10, and I'm not sure if you're working with the new Windows Terminal or the older Command Prompt. If you're using the older Command Prompt without any issues, this might be a Windows 11 or Windows Terminal-specific issue.
It seems that the demo works with no problem if I don't use the new Windows Terminal applications
After some debugging, I think this issue is causes by the usage of package pyreadline3
. Here is my debugging process.
First of all, this issue only occurred when use_rawinput=True
, so I found this in cmd2
:
At first I suspected the issue is somehow caused by configure_readline()
and restore_readline()
, so I try delete these two function calls, however that didn't work.
Then I started to check this input line:
https://github.com/python-cmd2/cmd2/blob/3062aaa195e1afac2b9058902bb4530b6668787a/cmd2/cmd2.py#L3239
I used the debugger to step into the execution of this input()
function, and it went into the pyreadline3
library, below is the call stack.
_update_line (<pkgs_path>\pyreadline3\rlmain.py:535)
readline_setup (<pkgs_path>\pyreadline3\rlmain.py:602)
readline (<pkgs_path>\pyreadline3\rlmain.py:605)
hook_wrapper_23 (<pkgs_path>\pyreadline3\console\console.py:842)
read_input (<pkgs_path>\cmd2\cmd2.py:3006)
_read_command_line (<pkgs_path>\cmd2\cmd2.py:3053)
_cmdloop (<pkgs_path>\cmd2\cmd2.py:3136)
cmdloop (<pkgs_path>\cmd2\cmd2.py:5285)
main (<cwd>\main.py:105)
The input()
function triggered the hook_wrapper_23
in pyreadline3
:
The hooks above triggered readline_hook
function. (At this point the readline_hook
is a pyreadline3.rlmain.Readline
object)
Then we reached Readline.readline_setup()
, and this setup function calls two functions:
def readline_setup(self, prompt=""):
BaseReadline.readline_setup(self, prompt)
self._print_prompt()
self._update_line()
It seems that the _update_line()
is in charge of updating line and control the scroll behaviour when a new line has been entered. And I went into _update_line()
:
def _update_line(self):
c = self.console
# ...
if (y >= h - 1) or (n > 0):
c.scroll_window(-1) # <--------
c.scroll((0, 0, w, h), 0, -1) # <--------
n += 1
Here two scroll-related functions has been called. Then I checked the scroll()
function:
def scroll(self, rect, dx, dy, attr=None, fill=" "):
"""Scroll a rectangle."""
if attr is None:
attr = self.attr
x0, y0, x1, y1 = rect
source = SMALL_RECT(x0, y0, x1 - 1, y1 - 1)
dest = self.fixcoord(x0 + dx, y0 + dy)
style = CHAR_INFO()
style.Char.AsciiChar = ensure_str(fill[0])
style.Attributes = attr
return self.ScrollConsoleScreenBufferW( # <------- Here
self.hout, byref(source), byref(source), dest, byref(style)
)
As the docstring suggests, """Scroll a rectangle."""
, this function seems to handle scrolling behavior within a defined rectangle, which aligns with the issue I'm facing.
I looked into the console function used in scroll()
, ScrollConsoleScreenBufferW
, and found relevant information in the Microsoft Console API documentation.
The documentation describes the behavior of this API as follows:
Moves a block of data in a screen buffer. The effects of the move can be limited by specifying a clipping rectangle, so the contents of the console screen buffer outside the clipping rectangle are unchanged.
I believe this API is causing the issue. Additionally, the PSReadLine GitHub PR seems also removed the usage of ScrollConsoleScreenBuffer
to resolve a similar scrolling issue.
The one used in
pyreadline3
has an extra alphabetW
, and I'm not sure if it's related to theScrollConsoleScreenBuffer
in the Microsoft API docs.
After having all the info above, I tried to replace the scroll()
and scroll_window()
with console.write('\n')
and this time the scroll behaviour becomes the one I want:
# pyreadline3/rlmain.py
class Readline(BaseReadline):
# ...
def _update_line()
# ...
if (y >= h - 1) or (n > 0):
# replace scroll and scroll_window with a single console.write('\n')
c.write('\n')
# c.scroll_window(-1)
# c.scroll((0, 0, w, h), 0, -1)
n += 1
Also, the comment in pyreadline3
says that one extra line is preserved for IEM statusbar. And I guess this is the reason why there is one extra empty line at the bottom when set use_rawinput=True
in cmd2
:
use_rawinput = True
use_rawinput = False
If we just let the program ignore this (change the diff condition y >= h - 1
to y >= h
), those two scroll functions will not be triggered and the issue is also resolved.
# pyreadline3/rlmain.py
class Readline(BaseReadline):
# ...
def _update_line(self):
c = self.console
# ...
x, y = c.pos() # Preserve one line for Asian IME(Input Method Editor) statusbar
w, h = c.size()
# if (y >= h - 1) or (n > 0):
if (y >= h) or (n > 0):
# after changing the condition, the demo python program
# will never reach inside this if block, thus no more
# scrolling issue.
c.scroll_window(-1)
c.scroll((0, 0, w, h), 0, -1)
n += 1
These are two possible workarounds for my specific case. However, I am currently unable to find the approach to resolve this issue.
One possibility we might be able to explore would be to detect if the cmd2 application is running in Windows Terminal
and if so not use pyreadline3
.
My understanding is that the new Windows Terminal
is pretty much a POSIX-compliant terminal complete with support for things like ANSI escape codes and the like. I don't have Windows to experiment, but I think there is a good chance stuff would "just work".
@kmvanbrunt Based on all of the data provided by @nfnfgo do you have any smart ideas?
Version Info
Here is my version info.
python: 3.12.4
cmd2: 2.4.3
windows: Windows 11
powershell: 7.4.5
Issue
I'm using
cmd2
to create an interative CLI on Windows. And as the title implies, It seems there is terminal output issue whenuse_rawinput=True
.Concretely, if the output lines count of
cmd2
application is over the vertical size of the Terminal on Windows, the overflowed line will directly disappeared from the terminal console history. A video is attached below to demostrate the issue:https://github.com/user-attachments/assets/1b3fc70f-1468-476a-86fc-c85b5ec516cd
The video is using Terminal application on Windows, but the integrated terminal in VSCode could also reproduct this issue on my computer. The code used is also attached below:
Code used in the video
```python import sys import os from typing import Optional, List, Iterable import cmd2 from cmd2 import Settable, Statement, utils from cmd2 import ( Cmd2ArgumentParser, with_argparser, with_argument_list, with_default_category, ) class ExampleCmd2Application(cmd2.Cmd): def __init__(self): self.use_rawinput = True super().__init__(startup_script=".sudokurc", silence_startup_script=True) self.count = 0 def do_iter(self, *args, **kwargs): self.poutput(self.count) self.count += 1 def main(): app = ExampleCmd2Application() app.cmdloop() if __name__ == "__main__": main() ```This issue disappeared once I change the code above into:
After some simple investigation, it seems that this issue is related to either readline, Windows Powershell or Windows Terminal, following are some relevant links:
https://github.com/microsoft/terminal/issues/10975#issuecomment-901480065 https://github.com/PowerShell/PSReadLine/issues/724
Based on the search result and the fact that I failed to reproduce this issue in GitHub Codespace in Linux environments, I assumed this is a platform-specific issue which only exists on Windows.