[BUG] `get_markup` sees 2-bit colors as 8-bit ones

[jeffwright13]

When I try to display ANSI text in a Window object, the colors are all wrong: no red or yellow ; cyan shows up as green; everything else looks blue. Any clue why this is happening?

How it looks on terminal:

How it looks in PTG:

Source code:

from __future__ import annotations
from pytermgui.parser import MarkupLanguage as ML
import pytermgui as ptg

summary_results = """============================ 16 failed, 16 passed, 5 skipped, 6 xfailed, 3 xpassed, 22 warnings, 9 errors,  in 12.84s ============================="""

def _configure_widgets() -> None:
    ptg.boxes.SINGLE.set_chars_of(ptg.Window)

def _define_layout() -> ptg.Layout:
    layout = ptg.Layout()
    layout.add_slot("Header", height=1)
    layout.add_break()
    return layout

def main(argv: list[str] | None = None) -> None:
    _configure_widgets()
    with ptg.WindowManager() as manager:
        manager.layout = _define_layout()
        header = ptg.Window(
            ML().get_markup(summary_results),
            # "[app.header] Welcome to PyTermGUI ",
            box="EMPTY",
        )
        manager.add(header)

if __name__ == "__main__":
    main()

[bczsalba]

I think the issue is related to the library parsing something like \x1[31m as and xterm color with the index of 31, and not the first 2-bit color available. Probably can be fixed somewhat easily, though it has given me some headaches before!

Anyways, where does this string come from? If you can control it, I would highly recommend rolling your own markup equivalent of it, as that is gonna be a lot more readable and extensible later on. I'll try to fix the issue in the meanwhile.

[jeffwright13]

The ANSI string comes from pytest. It is the same string sent to the console/terminal when it prints its output. I basically have a tee function implemented that saves to file the exact same strings sent to the terminal, so I can replicate in the TUI what the use would see in a standard pytest session.

Here's a full copy of the output from a pytest session if you want: https://drive.google.com/file/d/17QbBmBKY7umQIk0rc9BnIbSvS04SIS1N/view?usp=sharing

[bczsalba]

The string you provided earlier seems to work fine now. There were a couple of major architectural problems, but they should now have been resolved.

I also looked at the file you provided just now, and it mostly works but not fully. This is because pytest apparently uses a very nuclear reset of \x1b[39m\x1b[49m\x1b[0m, instead of just \x1b[0m which I think should do the same by standard. There is also the issue of TIM seeing [ and becoming very happy to parse it, so if you go this direction I suggest using something like this to "sanitize" your text:

ptg.tim.print(
    ptg.tim.get_markup(
        # I trust that pytest know why they need all 3 sequences, so we convert them to something TIM can parse
        content.replace("\x1b[39;49;00m", "\x1b[39m\x1b[49m\x1b[0m")
        # TIM would parse certain things that start with `[`, so we need to escape them
        .replace("[", r"\[")
        # We also escaped all CSIs (`\x1b[`) in the previous line, so we need to re-add them
        # You can probably have a much more syntactic result if you use some regex here, but this works too.
        .replace("\x1b\[", "\x1b[")
    )
)

Thanks for the report!

[jeffwright13]

Interesting. Pytest is sending a control code to set foreground and background colors to "default", then sending a reset command. Like you said, I'm sure there's a reason they do that. There are probably tens of thousands of terminals running this code right now, so obviously it works.

Anyway, re: the issue at hand. I can confirm that the changes you just made fix the color issue in the final "short test summary" line. Here's a screenshot for proof. Oh, and look at that sweet dummy pytermgui TUI (coming soon to a release near you!):