[BUG] Screen updates cause lots of scrollback buffer to be created

[baragona]

Describe the bug Screen updates, such as animations, produce lots of output in the scrollback buffer.

To Reproduce Perform almost any action in pytermgui application and notice how the scrollbar gets really large, and you can in fact scroll back to see the history of what the screen looked like. This was done with iTerm2.

Expected behavior Most ncurses based applications do not seem to spam the scrollback buffer this much.

System information This is on macos Monterey with iTerm2. Running python inside of docker.

PyTermGUI version 6.1.0

System details:
    Python version: 3.9.10
    $TERM:          xterm
    $COLORTERM:     None
    Color support:  ColorSystem.STANDARD
    OS Platform:    Linux-5.10.104-linuxkit-x86_64-with-glibc2.31

[jeffwright13]

I've observed this as well. Python 3.9.9, Mac Monterey 12.3.1, iTerm 2 Build 3.4.15.

[bczsalba]

I've also seen this happen, but not sure why. Theoretically, setting an alternate buffer should mean that no scrollback is preserved after exit, but iTerm doesn't seem to respect this for some reason.

[SomwareHR]

Same problems here

Windows Terminal 1.12.10334.0

• Animations visible
• Scrollback visible
• Interaction with TUI not possible

System details: Python version: 3.9.1 $TERM: xterm-256color $COLORTERM: None Color support: ColorSystem.EIGHT_BIT OS Platform: Windows-10-10.0.19041-SP0

---

ConEmu v.220418

• Animation visible
• No scrollback
• Interaction not possible

---

WSL 2

• Animation visible
• Scrollback visible
• Interaction is possible here

System details: Python version: 3.8.10 $TERM: xterm-256color $COLORTERM: None Color support: ColorSystem.EIGHT_BIT OS Platform: Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with-glibc2.29

Edit:

Windows 10 CMD

This one throws pure ANSI codes out.

I didn't check on Linux machine yet. Seems that this module is not Windows-friendly 😃

[bczsalba]

@SomwareHR yup, at the moment the library's windows support is more "this works? cool" than "this has to work". I elaborated on this in some previous issues, but the basic problem is that I don't have a Windows machine I can develop on, so testing is pretty difficult.

It will work on both MacOS and Linux, though! I'm not sure what the Windows Terminal team wants to achieve, but they seem to have implemented a lot of the xterm standard into the program, where historically Windows kinda just did its own thing. If they implement xterm-compatible mouse support before I (or another contributor) implements it for Windows then that should work as well.

[Tired-Fox]

I found something that clears the buffer and doesn't create scrollback. It is a bit more manual, but it works.

The codes should be the same for Linux and Windows so this could be a universal fix for scrollback.

@bczsalba Let me know if this is something you want to integrate into the library

def clear_buffer(buffer: TextIO, height: int):
    """Go to top left of buffer and delete all lines. 
    It will delete the first line and up to terminal height.

    Args:
        buffer (TextIO): The buffer to manipulate
        height (int): The height of the buffer
    """
    buffer.write("\x1b[0;0f")  # Moves to top left
    buffer.write(f"\x1b[{height}M")  # Delete lines

This will delete all lines from the buffer and put the cursor at the start of the buffer.

Average Times:
    Sec  : 2.670998976100236e-06 sec
    Micro: 2.670998976100236 μs
    Nano : 2670.998976100236 ns

[bczsalba]

@Tired-Fox

@bczsalba Let me know if this is something you want to integrate into the library

The current screen clear method is quite simple (just writes \x1b[2J to stdout I'm pretty sure) so we could probably include this somewhere.

I think this can be done in ansi_interface.clear() either included in screen, or as a separate argument (maybe buffer?). We could also add an argument for screen+buffer that does both.

P.S. in the line:

    buffer.write("\x1b[0;0f")  # Moves to top left

You are probably better off using \x1b[H, as it takes the cursor "home", e.g. the origin of the terminal (0, 0 on Windows, 1, 1 on UNIX-like).

---

As mentioned in #81 I'm working on a compositor that updates the screen char-by-char, which (I think) would fix this issue anyways. We should still include something for this, as the whole idea behind PTG is that the WindowManager isn't meant to be the only way to do things.

[Tired-Fox]

I can look at a couple spaces and make some ideas.

[jerch]

On a sidenote: ED (erase in display, CSI 0|1|2|3 J) is meant to clear content of the currently visible terminal area, not to scroll it off. For some reason some terminals implement full viewport erase (CSI 2 J) as a scroll off on normal buffer. Basically this is a mistake of the terminal, as for scrolling line content there are other sequences like SU/SD.

[bczsalba]

This issue is related to #81, so the note I left there is relevant here as well. There is a way to do this better (specifically diff-based screen updates), but in order to implement that we have to change pretty much every bit of PTG's underlying concepts, which is one step short of a full rewrite. This full rewrite has already been done under a new project, so if this bothers you I recommend checking out Celadon and its application framework celx :)