DomTerm freezing - no input or output

[mcarans]

Recently, I have found that DomTerm has started to hang frequently. Today, I tried upgrading to the 4.2 stable libwebsockets and the latest DomTerm from GitHub (which shows as 2.9.3 in the About menu), but the problem persists.

The terminal window freezes and no further input or output is possible. The only thing I can do is close and reopen. This has unfortunately rendered DomTerm unusable for me because it happens quite often. Any help would be appreciated.

[PerBothner]

This could be a bug in the flow control: The count of bytes send from the backend (server) to the front-end (browser) is off, so the backend waits for confirmation that never arrives.

I can try to reproduce it if you have a repeatable way of forcing the problem. If is just "needs to run for a while" it may be more difficult.

A start would be to start domterm with the -d 15 flag. Look for messages containing "flow control" in the resulting log file (/tmp/domterm-PID.log by default). If the last such message starts with "session SS paused (flow control)" then it is almost certainly a flow control problem. It should also have text including "MMM bytes ahead". If MMM is a really large number (indicating wrap-around) that suggests more bytes confirmed by the browser than than the running count kept by the server. Otherwise, if MMM is a "normal" number that indicates some bytes were counted by the sender (server) but not the receiver (browser). (The implementation is a bit tricky because some "out-of-band messages" sent by the server should not be counted for reasons I won't go into here.)

[mcarans]

Here is an example of the freezing:

Here is the log. I see session paused and later session unpaused. I don't see "bytes ahead". domterm-9035.log

Does this help?

[PerBothner]

The browser is sending a lot of FOCUSED events to the server. I fixed a problem relating to that June 25. I'm guess either it wasn't quite the right fix (i.e. it doesn't work in your situation) - or somehow you're running an old server. When you type domterm it will connect to an existing server if one is running. However, I don't see how the About menu can report 2.9.3 unless it has the June 25 fix.

However, for sanity, please try the domterm status command. If should report the version of the currently executing server, if there is one. Also, exit all DomTerm windows and check ps aux|grep domterm or similar to make sure there isn't some domterm process hanging around.

[mcarans]

I checked and there are no DomTerm processes hanging around.

> domterm status
DomTerm version 2.9.3 (git describe: 2.9.3-4-gcb3c928)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.0-v4.2.0-82-gc042ba8c
Reading settings from: /home/mcarans/.config/domterm/settings.ini
Backend pid:14665, command-socket:/run/user/1000/domterm/default.socket
Window 1:  qtwebengine: 5.12.8, chrome: 69.0.3497.128
  session#1: pid: 14722, tty: /dev/pts/1

(when run from DomTerm)

> domterm status
DomTerm version 2.9.3 (git describe: 2.9.3-4-gcb3c928)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.0-v4.2.0-82-gc042ba8c
Reading settings from: /home/mcarans/.config/domterm/settings.ini
(no domterm sessions or server)

(when run from Gnome-Terminal)

[PerBothner]

Could you try the attached patch?

focus-patch.txt

[mcarans]

It still hung unfortunately. Here is the log: domterm-21712.log

[PerBothner]

It looks like the application process (i.e. apt-get) is writing bytes to the pty like crazy - look at all the RAW_RX pty 9 session 1 read messages.

One possibility is a bug (or incompatibility) in the libwebsockets library. I looks like you're actually using a newer version that I'm using. I'll update my clone.

One thing to try is the AppImage here - see instructions under Binary releases here.

I run Fedora, so I can't test apt-get but if there is some other simple way to reproduce the problem that would be great. But you can wait until that until I've tried updating libwebsockets.

[PerBothner]

I re-built libsockets. I did get a freeze one time, but wasn't able to repeat it so far. (Could have been network problem.)

Perhap try domterm with these flags:

domterm -d 15 log.js-verbosity=2 log.js-to-server=yes

[mcarans]

I tried with those flags. Here is the log after the hanging: domterm-47879.log

[PerBothner]

Nothing suspecious yet, but I'm still studying the lof, Have you tried using the DomTerm-2,8,3.AppImage? Probably won't make a difference but may increase or decrease the possibility the libwebsockets version may be an issue.

[mcarans]

I just tried the AppImage and it hung. I ran domterm status for it and it gives:

DomTerm version 2.9.3 (git describe: 2.1-604-gcc70999)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.99-v4.2.0-58-gbbd12d95
Reading settings from: /home/mcarans/.config/domterm/settings.ini
Backend pid:53031, command-socket:/run/user/1000/domterm/default.socket
Window 1:  qtwebengine: 5.12.8, chrome: 69.0.3497.128
  session#1: pid: 53111, tty: /dev/pts/1

I can get DomTerm to hang regularly. All I have to do is uninstall and reinstall something, I guess because apt remove or install generates a lot of output including a progress bar.

[PerBothner]

Could you try the AppImage with -Bwebview and/or a browser: -Bfirefox, -Bchrome-app, or plain -B (short for -Bbrowser), say?

[This is probably not relevant to the original "freezing" issue - but would be helpful to know if it's a Qt-specific problem.]

[mcarans]

-Bwebview seems not to have the problem. So it appears to be Qt-specific.

DomTerm version 2.9.3 (git describe: 2.1-604-gcc70999)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.99-v4.2.0-58-gbbd12d95
Reading settings from: /home/mcarans/.config/domterm/settings.ini
Backend pid:53931, command-socket:/run/user/1000/domterm/default.socket
Window 1:  gtk: 3.24.20, appleWebKit: 605.1.15
  session#1: pid: 53999, tty: /dev/pts/1

What version of Qt are you using? Mine is 5.12.8.

[PerBothner]

To make sure I'm not misunderstanding:

(1) You can run the AppImage with -Bwebview, and it starts up.

(2) But does it still freeze if you run apt?

(3) If you run the AppImage with -Bqt it freezes on/before startup?

(4) If you start domterm with -Bwebview using the domterm from git clone+configure+make - and then run apt, it still freezes?

[mcarans]

(1) I have not had any issues with starting DomTerm whatever the version. (2) With -Bwebview, it doesn't freeze with apt (3) With -Bqt, it freezes with apt (not before or on startup) (4) I get "cannot find dt-webview command" - I guess I'd have to go and set that up, but it does look like a qt problem given 2 and 3.

[PerBothner]

On my current laptop domterm status reports qtwebengine: 5.15.2, but I built the AppImage on a laptop with an older version of Fedora and hence an older version of Qt. (I can check later.) Ny hope was that linking against an older version of Qt might be more portable - but it doesn't shock me that a Qt app linked on Fedora 30 might not work on your version of Ubuntu/Debian/???. The webview version links against Gtk C libraries which is likely to be more portable (in terms of AppImage binary portability).

[PerBothner]

"cannot find dt-webview command" means you did not configure --with-webview.

[mcarans]

I wonder if I should try 5.15.2? BTW, is DomTerm compatible with Qt6?

[PerBothner]

I can't come up with a reason why the Qt front-end would freeze on apt but the webview version doesn't. But I can try some stress-testing with the Qt front-end.

I have not tested Qt6. There does seem to be Qt6 packages available for Fedora, but I haven't looked into how to get them - or probably just wait until Fedora 35 (?).

[PerBothner]

I don't know how difficult it would be for you to try Qt 5.15.2, but unless it is a simple update - or one of the other front-ends is too deficient or unappealing - I'd recommend -Bwebview or -Bchrome-app or -Belectron as easier to get working and reasonably pleasant.

[mcarans]

I tried QT 5.15.2 but same issue. domterm-28363.log

DomTerm version 2.9.3 (git describe: 2.9.3-7-g0572dd9)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.0-v4.2.0-82-gc042ba8c
Reading settings from: /home/mcarans/.config/domterm/settings.ini
Backend pid:27908, command-socket:/run/user/1000/domterm/default.socket
Window 1:  qtwebengine: 5.15.2, chrome: 83.0.4103.122
  session#1: pid: 27991, tty: /dev/pts/1

I've noticed that the hanging occurs when the text reaches the bottom of the window when I run "sudo apt install xxx". However, if on opening DomTerm, I hold down Enter until going past the bottom of the window and then run "sudo apt install xxx", there is no problem.

[PerBothner]

I did get it to hang with Qt, but unfortunately I didn't have logging on. However, domterm status did say "paused" which suggests a flow-control issue - but that doesn't match the log. But I'll keep the Qt window running and using it.

[mcarans]

If I run domterm status in another gnome-terminal window when DomTerm is hanging, this is what I get:

> domterm status
DomTerm version 2.9.3 (git describe: 2.9.3-7-g0572dd9)
Copyright 2021 Per Bothner and others
Using Libwebsockets 4.2.0-v4.2.0-82-gc042ba8c
Reading settings from: /home/mcarans/.config/domterm/settings.ini
Backend pid:35622, command-socket:/run/user/1000/domterm/default.socket
Window 1:  qtwebengine: 5.15.2, chrome: 83.0.4103.122
  session#1: pid: 35706, tty: /dev/pts/1

I don't see "paused".

A hack for my case would be if there was a way to open DomTerm and automatically send say 20 carriage returns to get past the bottom of the window. I'm not sure if that's possible though.

[mcarans]

I am able to open another tab in the DomTerm window and type commands while the first tab is frozen, but the second tab will also become frozen on hitting the bottom of the window.

Note that for your testing, once you have scrolled past the bottom of the window, then the problem seems to go away, so you will need to open a new tab or window.

[PerBothner]

The 2 times DomTerm froze for me was bottom of the window, IIRC, but I've hit bottom of window multiple times with no freezing.

Any strong reason you prefer -Bqt over -Bchrome-app, -Bwebview or -Belectron? If there is, maybe it is something I can help you with. I can see a Mac user might strongly prefer the "real menus" of -Bqt or -Belectron (because how MacOS uses a global menubar), but otherwise the "JavaScript menus" should hopefully be OK (and have the advantage that multi-key shortcuts are displayed). An important issue is Copy/Paste; let me know if you need help with that.

[mcarans]

Good news. I've discovered a fix of sorts. If I change my settings.ini from geometry = 1280x600 to geometry = 1300x620 that seems to fix the problem somehow.

I think it's the progress bar of apt being at the bottom of the screen that caused the problem.

I assumed that the Qt frontend would be the most performant since it is C++ rather than Javascript.

[PerBothner]

"If I change my settings.ini from geometry = 1280x600 to geometry = 1300x620 that seems to fix the problem somehow."

That is weird. Perhaps relating to fractional characters/clipping? Or possibly line-breaking?

"I assumed that the Qt frontend would be the most performant since it is C++ rather than Javascript."

The Qt front-end that is in C++ is really minor - mostly menu support and a few other things. I don't think there is anything critical-path. Communication with the backend/server (which is written in C++/C) is primarily using WebSockets, using JavaScript APIs (on the frontend side) that are independent of the front-end. Of course it is possible there may be a performance difference between web engines based on Firefox/Gecko, Chrome, Safari/Webkit, for different tasks - but a given version of the Chrome engine is likely to perform more-or-less the same whether it is Electron, Qt, Edge, or the Chrome browser (e.g. -Bchrome-app).

The WebView front-end on Linux is just a thin layer over GtkWebKit (which is based on the Safari engine). In the future, I'm more likely to spend more time polishing that (or some other GtkWebKit wrapper) than the Qt front-end.

[PerBothner]

"I think it's the progress bar of apt being at the bottom of the screen that caused the problem."

Plausible - however, the 2 times it froze for me, I don't believe there was a similar progress bar.

[mcarans]

Thanks, good to know the direction you are taking. It will be easier not to have to set up Qt. It would be good to update the installation instructions to reflect that WebView is the preferred frontend. I think it currently says Qt (with WebView listed as Experimental) while on the DomTerm homepage, it says Electron is the preferred frontend.

[PerBothner]

It is not clear which front-end is to be preferred/recommended - and it changes - and might be different for different people and platforms. (Specfically, Mac is quite different.) I am trying to reduce front-end-specific code to whatever is really worth it. Recently, I've changed it so all front-ends use iframes for secondary panes (before there were problems using iframes with the Javascript menus), and supported more solutions for copy/paste handling to work around browser security blocks.

Another problem I recently noticed with Qt that I don't have a solution for: Fullscreening causes some nasty behavior requiring manual process killing. Could be a Wayland issue.

If you feel inspired to test out various front-ends, I'd welcome your feedback and experience. As far as I know this page is mostly correct, but could probably do with some updating and reorganization - as could other parts of the documentation.

[mcarans]

@PerBothner Just tested Electron, webview and --chrome-app and all have the same problem as with Qt.

Here is my settings.ini:

shell.default = /bin/bash
command.electron = /home/mcarans/Programming/electron/electron
geometry = 1280x600
style.dark = on
style.user =
 |div.domterm { --monospace-family: "JetBrains Mono","Monospace","FreeMono"; font-size:14px }
 |div.domterm {  --main-light-color: white; --main-dark-color: black }
 |div.domterm-spacer {background: none }
 |div.domterm {--dt-blue: #729FCF; --dt-green: #8AE234; --dt-yellow: #FCE94F; --dt-red: #EF2929}

I also tried commenting out: source "$BASH_IT"/bash_it.sh in my .bashrc to remove bash-it's changes (bash-it is a theming framework for bash). The error still occurs without bash-it being loaded (ie. no bash theming).

The bad news is my resolution change to 1300x620 doesn't seem to help any more. I had pulled latest changes. My latest log is here: domterm-23848.log

[PerBothner]

So far no hanging with those settings.

I did check in some small fixes, but I don't think anything that would explain the problems.

I'll keep (and maek use of) the window for a while.

[mcarans]

@PerBothner The hang only occurs when you first open DomTerm and the text reaches the bottom of the window for the first time. If the text manages to get past the bottom of the windows without hanging, then it will work fine thereafter. So one manual fix is for me to open DomTerm and press Enter 20 times to move to below the bottom of the window - then I can use DomTerm fine thereafter.

[PerBothner]

Even stranger.

Does it hang after a clear and reaching the bottom again?

[mcarans]

Yes it does. You can see here what the window looks like when it hangs. The left and bottom edge of the window turn orange (QtDomTerm).

[PerBothner]

Any chance this helps?

sregion-patch.txt

[mcarans]

Unfortunately not - same issue.

[PerBothner]

How about if we do something that makes slightly more sense:

var top = Math.max(this.getParameter(0, 1), 1);

(What I meant to do - hopefully.)

[mcarans]

That fixed it! (I applied the patch and then the line var top = Math.max(this.getParameter(0, 1), 1); on top of the patched file - I wasn't sure if you meant that I revert the patch and apply only the line.)

[PerBothner]

Great! I noticed the application was trying to set the scrolling region to lines 0 up to 32 - but it should be 1 up to 32 (since line numbers start at 1 in escape sequences).

[PerBothner]

Somebody (maybe you?) mentioned that telnet mapscii.me wasn't working. It should work now.

[mcarans]

@PerBothner Wasn't me who mentioned telnet.

Thanks so much for fixing this freezing issue!

[PerBothner]

Found the comment about telnet mapscii.me - it was in issue #99.