Some texts got disappeared from the terminal

[Souravgoswami]

I was browsing through the help section of irb to see the available prompts I can use. I was running everything locally. I was trying modes, scrolling up and down the few lines in the terminal. Then I found out that the modes aren't there. It seemed to me like a bubble universe!!

It's so weird. For example, take a look here:

• This is original (newly spawned terminal):

• This is what I got after like 10 seconds of scrolling:

---

Right the text disappeared. So I can't find them with ctrl + shift + f either.

I thought I am probably a victim of the Mandela effect! Did I see the valid --help section in dream!? But it took me a minute to discover what's going on!

I never witnessed such terminal error. It could be daunting and perhaps it could give a heart attack to devs debugging large apps. So I just let you guys know.

$ tilix --version
Versions
    Tilix version: 1.9.1
    VTE version: 0.62
    GTK Version: 3.24.24

Tilix Special Features
    Notifications enabled=0
    Triggers enabled=0
    Badges enabled=1

[egmontkob]

Another similar report at https://askubuntu.com/q/1479000, making me strongly suspect that it's a bug in irb rather than in tilix.

[egmontkob]

I just noticed that over there the poster shows the screenshot of konsole, a totally different terminal emulator, facing missing lines in irb's output.

I'm pretty certain it's a problem with irb. Maybe it sets up an explicit scroll region (sets of rows so that scrolling occurs within them only).

How reliably can you reproduce the problem? What steps do you execute then? Can you reproduce with, say, xterm or konsole (or some other terminal that is not based on VTE)?

[Souravgoswami]

It's possible, though quite unusual, that the issue lies with IRB using ANSI escape sequences. Could it be that IRB outputs a specific sequence that causes glitches in terminals? If so, this would suggest a bug within the terminal itself.

Furthermore, when IRB ceases execution, the text shouldn't vanish. There's no code within IRB designated to erase lines of text. This leads to the suspicion that the terminals themselves might be exhibiting a bug when interpreting the ANSI sequences produced by IRB?

[Souravgoswami]

This glitch happens intermittently, unfortunately I can't reproduce it currently :(

Regarding your question on trying something other than VTE: I never faced this issue on terminals like Alacritty. But this is very difficult to say with certainty as this error is completely random :(

[egmontkob]

But how likely is it that multiple independent terminal emulators suffer from the same bug? I mean, it's possible, but I think it's way more likely that irb emits broken sequences.

Can you still reproduce the bug? Because I could not.

If so, could you please run it under script, and then share the resulting typescript file? We'd need to examine the sequences manually to verify if the terminal executes them properly.

(We're cross-posting, I typed these before your previous comment appeared.)

You posted your bug a long time ago, it's easily possible that irb has fixed it since. Would be interesting to check their changelog and bugtracker, maybe we can easily find this issue and fix.

[egmontkob]

This looks relevant to me: https://github.com/ruby/irb/issues/175

[egmontkob]

Okay, I can reproduce some corruption using irb now.

I'll investigate a bit later today and get back to you with my findings.

[Souravgoswami]

Ah, I see now. It's possible there's an issue with the escape sequences used by irb and how VTE-based terminals interpret them.

Interestingly, the script command isn't capturing the output from irb. When I execute irb --help, the output isn't recorded in the typescript file. Instead, reading the file only shows my prompt (PS1).

At the moment, I can't reproduce the issue. However, it would certainly be valuable to reproduce it for further investigation. I've updated my system multiple times, which might have resolved the bug. For reference, here are the current versions:

$ tilix --version 
Versions
    Tilix version: 1.9.5
    VTE version: 0.74
    GTK Version: 3.24.38

$ irb --version
irb 1.7.0 (2023-06-03)

Just noticed that sometimes the content of typescript is this:

In other words, the output is broken.

[egmontkob]

Ah, I see now. It's possible there's an issue with the escape sequences used by irb and how VTE-based terminals interpret them.

Yes, indeed. irb is outputting truly untypical escape sequences, and VTE doesn't handle the scrollback buffer like xterm does in that case.

I've filed upstream bug https://gitlab.gnome.org/GNOME/vte/-/issues/2663 for this.

Basically: an interactive session might break what's at the top of the window. Perhaps your irb --help was followed by an interactive session which retroactively broke irb --help's output?

Let's hope this is the only problem that caused all this trouble.

Interestingly, the script command isn't capturing the output from irb.

It surely captures everything, it just buffers it up before writing to the output file. Make sure to cleanly exit (e.g. ^D) script itself, too. Or use its -f flag (and don't enter an infinite loop by cat'ing typescript within the script session :)).

Thanks a lot for your report and your help in investigating! :-)

[Souravgoswami]

Thank you, Egmont, for shedding light on this intricate issue.

Regarding the script command, its behavior in my environment is quite peculiar. The output I'm getting is:

exit

Script done on 2023-10-04 16:53:54+05:30 [COMMAND_EXIT_CODE="0"]

Rest assured, I'm not caught in any infinite loop, and I've been conscientiously making a clean exit using CTRL + D.

In reference to the interactive session potentially disrupting the content at the top of the window:

Basically: an interactive session might break what's at the top of the window. Perhaps your irb --help was followed by an interactive session which retroactively broke irb --help's output?

I wholeheartedly agree with this observation. It seems to be the root cause of the anomalous behaviour.

Turning to the example you provided on https://gitlab.gnome.org/GNOME/vte/-/issues/2663:

$ seq 100
[ prints 1 .. 100 ]

$ print "\e[1S"

Upon execution, the number 80 does vanish from the sequence. I've experimented with various terminals including xterm, terminology, cool-retro-term, and konsole. These terminals seemed immune to the issue.

However, the problem is quite evident in gnome-terminal, XFCE terminal, tilix, and similar environments.

Thanks a lot for your report and your help in investigating! :-)

Your insights have been invaluable. A heartfelt thank you for your dedication in pinpointing the root cause. Your effort is deeply appreciated.

I'm eagerly looking forward to the resolution of this bug. Cheers to progress and collaboration! 🎉✨

[egmontkob]

I've experimented with various terminals including xterm, terminology, cool-retro-term, and konsole. These terminals seemed immune to the issue.

Interestingly, the linked AskUbuntu post shows a buggy screenshot of konsole. But it's a kind of "bug" that a terminal emulator can easily make and it's not that unlikely that some time ago konsole also used to suffer from this bug.

I said "bug" in quotes because strictly speaking I don't think there's any standard mentioning the scrollback buffer at all and defining what exactly needs to go there; it's more of a common sense thingy that xterm and friends are doing the right thing here and VTE needs to adjust.

However, the problem is quite evident in gnome-terminal, XFCE terminal, tilix, and similar environments.

These all use the very same VTE widget for terminal emulation. If it's buggy in one of them, it's buggy in all of them :)

[egmontkob]

Hi,

There's a proposed patch available in the upstream bugtracker.

Standalone test: grab VTE's source, apply the patch, compile with meson && ninja and launch the test app using ./src/app/vte-2.91.

If it works (it really should) then you can install the VTE library system wide, replacing your current one. Make sure to apply --prefix=... and friends to meson, and all the other compile/install flags according to your taste. Also note that your distribution might ship some patches which would be gone this way. I recommend that you recompile the very version shipped by your distribution and rebuild the package with this new extra patch (assuming that the patch applies easily), and install the resulting packages using your distro's package management software. Also have a non-VTE-based terminal (e.g. xterm) lying around, just in case you really mess up the installation. Good luck! :)

As per the upstream discussion, it's not entirely agreed upon that the desired behavior is to add these lines to the scrollback, and Windows Terminal is also known to suffer from this same issue. Because of this, they convinced the nutshell project not to rely on these escape sequences. Maybe irb should change their software too, to be on the safe side. If you feel like bringing it up with them, it would be highly appreciated.

[egmontkob]

Finally the fix has landed in VTE.

The next major stable release, VTE 0.76.0, the first containing this fix, is scheduled to be released in March 2024, along with GNOME 46. Until then, the development series tend to be actually quite stable and usable, for those who really need this fix :)

Unfortunately the fix (and a couple of prerequisite fixes that I decided to build upon) are complex enough that they are not a realistic candidate for a minor 0.74.x (or even earlier branch) release.