Incorrect Rendering Of Ncurses Style Borders

[fundamental]

Currently sub-windows borders created by ncurses don't render correctly. This is a problem that I've seen with a few of the other terminal capturing tools out there, though I was hoping that it wasn't going to appear here.

Example application:

Rendering in a recent version of chrome:

Would it be possible to add support for these terminal features?

[tweekmonster]

@fundamental I can't reproduce this. Here's my test with an ncurses program: https://share.esdf.io/JmD1FjKi09/ncurses.html

What confuses me is that in both of your screenshots, the long vertical pipes are rendered correctly but is missing in the left pane's content. I can tell this by the bulges in the line which are a side effect of anti-alias font rendering.

Would you be able to share the generated HTML? Maybe I can discern something from that.

[fundamental]

I'm confused as to how borders differ from normal vertical pipe characters as well, but the bug appears on my system none the less :-/. You might be able to replicate this using 'midnight commander' (mc) as that seems to also trigger the bug on my system.

The original screenshoted html is: test.html

[tweekmonster]

Well that wasn't too informative 😑

How about this: tmux capture-pane -t test:1.1 -Jep | gzip > output.txt.gz

Obviously use your own target though. Leave off the piped gzip if you want to confirm the output before uploading it.

[fundamental]

Here you are: output.txt.gz

[tweekmonster]

That wasn't too helpful either. I was hoping to see secret SGR codes that I didn't account for. I did find this however: http://emonkak.hatenablog.com/entry/2011/12/30/131055

Translation:

When used the TERM to screen in tmux, the border for use in, say, a ncurses become like a "qqqqqqqq". 
This environment variable fixed when the NCURSES_NO_UTF8_ACS is set to 0. 
Refer to the man of ncurses details of NCURSES_NO_UTF8_ACS.

In tmux environment variable to set the write as follows.

set-environment -g NCURSES_NO_UTF8_ACS 0

[fundamental]

I guess the ascii shift in/shift out characters are used to change the character set to represent the outline that ncurses produces (see ascii 0x0e and 0x0f). That doesn't entirely explain the broken newlines though...

[tweekmonster]

Okay yeah I see what's going on now. Took me a minute to see 0x0e and 0x0f once I had it in a hex editor. It looks like it's because 0x0e and 0x0f are included as characters instead of omitted. So, tmux2html is wrapping the lines due to the extra length. I think I could support this. Is there a translation table on the alternate character set I can use for reference?

[fundamental]

One of the docs I saw referred to it as the 'G1' character set which is an iso standard of some sort, but I'm not finding a good page describing the mapping.

Using a very simple C test program it seems that these shift in/out codes are the issue.

#include <stdio.h>

int main()
{
    int char_set=0;
    while(1) {
        int ch = getchar();
        if(ch & 0x80)
            return 0;

        if(ch == 0x0e) {
            char_set = 1;
            continue;
        } else if(ch == 0x0f) {
            char_set = 0;
            continue;
        }

        if(char_set && ch == 'x')
            putchar('|');
        else if(char_set && ch == 'q')
            putchar('-');
        else
            putchar(ch);
    }
}

I'll just see if there's some obvious utf8 characters for the vertical lines and corners.

[tweekmonster]

Found this: https://en.wikipedia.org/wiki/Talk%3AVT100#Alternate_character_set

In the case of k, it's 0x6b, so on that table, row 6, column b is ┐ which seems right.

[tweekmonster]

Update: Looks like i can put these characters in a table and get the coordinates with:

y = (ord(c) / 16) - 6
x = ord(c) % 16

[fundamental]

Nice, those values look right. What an odd bit of history that ncurses still uses.

[tweekmonster]

@fundamental Got it working. I'm just going over the script to make sure I didn't introduce a bug since I'm a heathen without unit tests at the moment. I'll close this issue once the changes are merged and pypi is updated. Thanks for helping me make tmux2html awesomer!

env LC_ALL=C sudo dpkg-reconfigure tzdata https://share.esdf.io/V9ao9L964k/ncurses-alt.html

Vs

env LC_ALL=en_US.UTF-8 sudo dpkg-reconfigure tzdata https://share.esdf.io/KSIfVsO76S/ncurses-utf8.html

[fundamental]

Thanks. This bug is entirely resolved after updating to the current version :+1: