Needs support for 256 color codes

[TooTallNate]

Hey @chjj, very nice project!

I have been trying to hack in support for 256 color codes which is IMO one of the only killer features missing, but all this crazy bit stuffing is making my head spin.

I'm dying to see Yoshi in my terminal in my browser :p

Expected behavior:

What is seen:

Cheers!

[chjj]

Hey Nate, thanks. This is something I've thought about a little bit.

One thing you should know is, all information about the characters in the screen buffer is packed into bits (highly modified from the original design): https://github.com/chjj/tty.js/blob/master/static/term.js#L1990

Here's an explanation of how the screen buffer and rendering engine work: https://github.com/chjj/tty.js/blob/master/static/term.js#L447

// In the screen buffer, each character
// is stored as a 32-bit integer.
// First 16 bits: a utf-16 character.
// Next 5 bits: background color (0-31).
// Next 5 bits: foreground color (0-31).
// Next 6 bits: a mask for misc. flags:
//   1=bold, 2=underline, 4=inverse

As you well know, unfortunately, javascript can only handle 32-bit ints when doing bitwise operations. As it stands right now, I set aside 5 bits each for the foreground and background colors.

Now, I could decide to only set aside 3 bits (although 4 would be just the same really) for the misc flags (the highest mask I'd need for it is 7). That would leave me 6 bits for each color - still not enough to reach even 88 colors; only 64 colors including the default fg and bg. I could maybe even push 7 bits each, if I dropped the flag for inverse, which isn't technically necessary. 7 bits would allow us to have 128 colors, which isn't too bad, but if we want 256 colors, we need 8 bits each. There's just not enough room.

The only way to support 256 colors would be to replace the current character attribute system with one that uses objects instead of packing data into bits. That would require a rewrite of a good amount of code, which doesn't bug me. I'm just worried that instantiating a ton of objects for each character in the screenbuffer would be costly performance/memory-wise, as opposed to just using a more lowlevel approach. I'm not sure if it would actually hurt the rendering speed. It would be nice to benchmark both approaches somehow.

I really really care about rendering speed. Right now on my machine, tty.js can render cmatrix seamlessly, which is probably the most control-sequence-intensive program you could ever dream of (aside from mplayer inside your terminal). tty.js uses a lot of cpu when rendering cmatrix, but it renders flawlessly without stuttering on my machine (at least in chrome it does ... it nearly crashes firefox because gecko/jaegermonkey just isn't fast enough). I consider that an accomplishment.

If you want to take a crack at replacing the attribute system with one that uses objects, that would be great. We could benchmark them side by side. :)

As an afterthought here ... just thinking out loud, one other workaround would be to add a flag which considers a character to be a "high color". Then we could set up a kind of abstract screenbuffer which maps to the original. This screenbuffer would only contain (high) colors, if they're necessary for the character. This might be a good way to avoid performance problems.

[TooTallNate]

We were able to get 256 color support in our fork of term.js by changing all the bit-stuffed Numbers to regular Objects with like "bold", "italic", and "underline", etc. I think our forks are too-far diverged for it to be able to be merged here, but I guess the moral is that is is possible.

As far as benchmarking goes, you could try recording a codestre.am and running "cmatrix", to get an idea of any performance differences. AFAICT though, there's no difference in speed (using any of the current modern browsers at least).

[chjj]

Very nice. I decided to implement it a different way just now. I borrowed you 256color generation code.

https://github.com/chjj/tty.js/compare/256_color https://github.com/chjj/tty.js/blob/256_color/static/term.js

I think it should be a bit faster, it just keeps the attributes and the character in separate numbers. Also, that term.js you're using is pretty old. A lot has been improved since then. I'd recommend switching to this one, or rebasing your commits on top of a more recent term.js.

I might merge this with master once I'm sure there's no noticeable performance hit.

[TooTallNate]

Very nice as well! Thanks for the tips about being outdated. I do like our Object approach though, since we we're now able to implement any number of character traits. We implemented "inverse" already as well.

[chjj]

Well, with this method, you still have 13 bits left to play around with in the character attributes. That should be enough for everything (blink, bold, underline, inverse, italic, etc). Anyway, even if you don't use this method, I'd still recommend rebasing your changes onto the current term.js. A lot has been fixed or improved since then.

[TooTallNate]

Ahhh, I was under the impression that you were out of bits to use from your description from before. We will indeed look into rebasing because as it is, yours adds some nice cross-browser support and stuff.

[chjj]

This change pretty much fixed every problem I mentioned above. I think the rendering (Terminal.prototype.refresh) would be a little faster than the object method, because you don't have to compare objects, you only have to compare numbers still. I'm probably going to merge it into master once I fool around with it enough, and convince myself it's not hurting performance.

[chjj]

I decided to merge 256 colors merged into master: 0e1c5f6bb58302ba7bedae42c887df1a73ca7b16 . I know a lot of people want it.

[TooTallNate]

Good stuff man!