Non Unicode Encodings

[NuSkooler]

First off, thanks for this library! The "Document" system is a lot like one I created (but mine is much less organized) for my BBS project. I had planned on re-doing the work, but I think using and extending upon terminal-kit makes a lot more sense.

This brings me to my question: I'd like to use terminal-kit for BBSing, and a MUD-like RPG I'm working on. The terminals involved are often much more basic than that of a modern terminal -- One of the main issues is encoding: "ANSI-BBS" terminals generally work using CP437 and not UTF-8. Is there a way to set I/O encodings with terminal-kit? What I do for other projects and libraries is using iconv-lite to convert to/from near the socket write/read.

[cronvel]

@NuSkooler I'm a bit lost. What is an ANSI-BBS terminal? I understand MUD RPG (it was one of my vaporware for more of a decade), but I'm have not played much with existing ecosystem. Do you mean that you directly SSH to the server, without any proper client?

[NuSkooler]

@cronvel https://github.com/NuSkooler/ansi-bbs-utils (which you may find interesting as some of the things I started working on there seem to be occurring in your lib as well) has some links.

Basically ANSI-BBS is a semi-standard of supported ESC sequences that BBSes serve. Old school ANSI.SYS and others did not have all of the modern ESC sequences we have now. For example DSR/CPR is often not supported. Modern Telnet clients include SyncTERM, NetRunner, MagiTerm, mTCP's mTelnet, etc.

The real question on this ticket is regarding encodings: From just poking around it seems that the I/O encoding is hard coded. I'd like to propose the ability to set the encoding -- e.g. for CP437.

[cronvel]

@NuSkooler Ok... To be honest, I'm open to a PR if it doesn't add some over-complicated layers, but I will not implement it by myself. I don't know what kind of escape sequence is supported, but I should warn you that Terminal Kit needs way more features than ANSI.SYS. The basic Windows terminal don't play well with Terminal Kit because of that.

For example, Terminal Kit needs to be able to request cursor position (send an ESC seq, and wait for another specific ESC seq from the terminal), something windows lack.

Luckily if you deal only with the document model, it needs way less features, because everything is backed by the ScreenBuffer.

You will need to create a termconfig for your terminal, add a hook system for the I/O.

[NuSkooler]

@cronvel I'm happy to work on this library and submit PRs. I'm not sure if you saw my email, but terminal-kit seems to do a lot of what my library in my BBS already does (but more generically) and what I had planned on doing as a more generic re-write. It makes more sense I think to work on the same project.

I've already created a ansi-bbs.js terminal info (very slapped together at this point). I will submit it when it's cleaned up.

DSR/CPR: I can live with it for now. Modern terminals all support this. Older DOS terminals do not, and some terminals have a limited "in queue" buffer (e.g. you can submit 10 requests, but until those are responded the 11th will not get a response). With the document model, it shouldn't ultimately be needed with a screen/paint buffer.

As for encoding: Do you have any hints as to where in the terminal-kit you think this would live/hook in? I will start looking at a implementation.

[cronvel]

At the moment, input management is done in Terminal.js at line 1060 (function onStdin()).

I have planned for a long time a full rewrite of the input + ESC seq parser, but haven't found the free time to do it. It would be done using the code found in the directory lib/vte/ (a work in progress VTE implementation, that would allow spawning a ScreenBuffer acting like a terminal).

[NuSkooler]

I'm still just getting to know this library, but peeking at onStdin, it's probably good to go.

For output encoding I'm having a hard time understanding all the spots in the code. This hack seems to mostly work, but it seems stdout and perhaps stderr are also written to?

conset iconv = require('iconv-lite');
// ...
this.term = TermKit.createTerminal({
    stdin   : this.socket,
    stdout  : this.socket,
    stderr  : this.socket,
    appId   : this.term.ttype,
    generic : this.term.ttype,
});

//  overwrite raw method prev bound to stdout.write()
this.term.raw = buffer => {
    const encoded = iconv.encode(buffer, this.encoding); //  encoding='cp437' for example
    this.socket.write(encoded);
};

[cronvel]

.raw() is only used by the ScreenBuffer. I think it's probably better to use transform streams, to make things more transparent. This way you just call the constructor with fake stdin/stdout/stderr.