Full support for DECSED,DECSEL,DECSCA, etc for xtermjs

[stuboydl]

Using xtermjs in VSCode to an application which respects VT220 protection sequences. DECSED sequence erases the complete screen including the protected fields. This is incorrect behaviour and currently means our developers are unable to test within the IDE.

Protected fields are normally protected from DESCSED erasure to support behaviour like clearing input data or updating areas without having to redraw the whole screen (maybe a throwback to 75 baud accousticly coupled modems?!).

Xtermjs InputHandler.ts currentlly says "Protection attributes are not supported".

Issue #2358 was closed on the (false) premise these sequences are unused. Any chance these could be implemented?

[Tyriar]

Could you provide a test case we could write to the terminal to use for this? I'm curious what other popular terminals (VTE, iterm2, Terminal.app) support this. If the support is not ubiquitous I'd still agree with this from #2358:

this gets a downvote as there was not a single request for this in over 4 years, no need to complicate terminal state with ancient unused stuff

It's just 1 request over 6 years now.

As for actual implementation if we wanted to do this, I think we could use the extended attributes system to avoid reserving an additional bit for every buffer cell, the terminal could also track a special mode to avoid checking the protection attribute unless it's used. Would need to learn more about it though.

[stuboydl]

@Tyriar We use the following terminal emulators which support it: Teraterm, Accuterm, Putty, OpenSSH. Plus there are more I know of.

I'm guessing there are more and more people using xtermjs through VSCode. In the "heritage" environment most of our guys work, they've been using simple line editors up until recently!! VSCode is opening up more environments for xtermjs to connect to.

The dump below is a typical screen from the system, I replaced the esc[ with "CSI" and put some newlines in to make it clearer. There's also a text copy of what it should look like at the end.

SCREEEN.txt

[jerch]

@stuboydl

Your app output looks very much like one of those mainframe/minicomputer app form interfaces of the 80s :smile_cat:. Wouldnt it be better to upgrade the app's drawing model to a more modern pattern? Because even if xterm.js would support protection flags, your app will most likely not handle arbitrary screen resizes (SIGWINCH) correctly, as hardware terminals did not have that flexibility. Whats the original terminal model the app was written for? (I assume that adding protection flags will not be enough to get the output fixed for you).

[stuboydl]

@jerch You are indeed near the mark, originally written in the 90s but still under active dev and maint; and yes, it would be better to rewrite for many reasons. However, sadly, that's not an option. Too many sites, too many dependencies, too much legacy, too few resources. Screen resizing is not an issue if the screen is generally at least 80x24.

[Tyriar]

This sounds like a case of the world moving forward and your legacy tools written 20-30 years ago should use legacy tools that support their quirks (back to the 1 request in our 6 years with the project). Reminds me of Internet Explorer and Edge's struggle to keep backwards compat, eventually culminating in an IE quirks mode in modern edge.

[stuboydl]

@Tyriar -1 for that response. Xterm was written 30-40 years ago and yet, you still support an implementation of it and, the sequences are still a part of the xterm protocol.

[jerch]

FWIW - it could be done with another bit in content, there is still plenty of room. Problem I foresee - it will suck quite bad on print performance, if done with branching on every single char (prolly should be loaded branchless on wcwidth-bitpos + 1).

Then only protected aware sequence handlers need to act upon it, all others including the renderer chain would not care at all.

[jerch]

@stuboydl Any chance to test the PoC PR yet? (#3712)

[stuboydl]

@jerch Sorry - been flooded lately (literally and figuratively). I will have a look later this week.

[theflyingape]

Hope this emulation feature makes it in. I have a group of developers still supporting an EHR system for a Boston-based healthcare system, and I've been recruiting them to use VS Code as their primary IDE and also for its versatile terminal support.

I heard "smooth scrolling" happened in VS Code already, both features used back in the mid 1980s off the new VT220s for their intended functional purposes. I would certainly add protected fields into my node xvt package, because it is used with xterm, et al.

[jerch]

@theflyingape Protection flag support is already done (see #4098).

On a sidenote - if those EHR systems originate from real DEC systems (e.g. VMS), most likely they will not run properly in xterm.js. Simply for the fact that DEC used to have very sophisticated libraries to build those form apps (e.g. DECforms and earlier incarnations). Those libs also make heavy use of the full sequence batteries of the old devices, where xterm.js implements only a small subsets. Ofc your success might vary with different apps, but I would not put to much hope into this endeavor.

[theflyingape]

Cool, will definitely be looking into this. Haha, no "FMS" used today, just wired in their apps that work for both proprietary Meditech and ported to classic VT, so we can use mainstream terminal emulators. But, yeah, did back in the BASIC Plus 2 days in manufacturing and later in architectural & civil engineering using VT340 Textronix for a CAD app. PCs took that over the next year.

[jerch]

Mind you - the power plants and healthcare systems argument crossed my way several times, but we never got a serious request in this regard. If I'd follow those arguments then Tektronix and ReGIS support is the real shot sh*t - well I am not against support for vector graphics in terminals, but imho reviving those ancient formats is the wrong approach. (I think the same way about sixel, but the sixel format already gained so much traction, that it cannot be ignored easily anymore.)

Regarding VT capabilities it is really amazing how far things were evolved back then. We (as "culture society") basically lost/forgot most of that with the GUI and web transition, putting us almost back to a typewriter-like interface on terminal emulator side (well more around vt100 caps, but still ignoring ~15ys of tech evolution). Fun fact - more recently we see a renaissance of the terminal idea, like ppl are rejecting web and GUI interfaces, instead choosing a more straight forward experience in terminal emulators. Back to the roots I guess. :smile:

And since 5-8ys most OSS emulators try to catch up and turn from cheapo typewriter-like interfaces into more advanced terminals (not talking of xterm here, it always was way ahead thanks to Mr. Dickey's dedication). On xterm.js side of things - we never aimed for full compat to VTxxx line (we dont even support proper VT level distinction), but rather went with whats still in use here and there. In supported sequences this makes xterm.js a mixture of all DEC lines (minus VT50 haha), plus some ANSI terminal features. Overall this might be only 15-20% of all sequences/features DEC ever came up with (just a guess, never counted it). This approach works surprisingly well even with curses apps, but ofc makes it almost impossible to run old DEC apps on it (which never was a goal, as there is low to no demand at all).

What may save you with old apps - if they are old enough they still might have a VT level 1 or 2 compat mode, where chances are high to get it running with xterm (xterm even up to level 3). Whether xterm.js can run those depends on the used sequences. For newer apps using VT level 4+ things are less likely to run in OSS emulators, as far as I am aware none resembles the much more complicated terminal setup DEC invented there (like left/right paging, multiple memories etc). You still might be able to get those running on some CS emulators claiming strict VT4xx+ compatibility though.

Sorry for the wall of text. :blush:

[theflyingape]

Love this curated project and the folks contributing -- very collaborative with thoughtful points of view. I wish more projects ran like this one.

I'm more of an integrator than a pure developer, such as DDgame and our hospital EHR classic using xterm.js, but I can appreciate every targeted point you took the time to make here. lol, yeah, sixel, been there, done that, and discovered it resurfaced too! Would you be surprised that I keep a MAME VT240 machine emulator using solcat for I/O to test stuff?

[jerch]

Would you be surprised that I keep a MAME VT240 machine emulator using solcat for I/O to test stuff?

Haha, well ~several months~ (no wait this was before corona lol) ago I took a trip through half of the country just to get my hands on a vt340 again, after idk 25ys? It was connected to a microVAX, both still in a working condition (VT had minor screen issues though). Sadly it was not for sale. :crying_cat_face: