Include CSS classes for 8-bit ANSI colors

[thiippal]

Describe the problem

I cannot get ANSI colours to show in Jupyter Notebook cell outputs.

I can see the ANSI colours in the Notebooks, and can render them successfully in HTML using nbsphinx.

The problem is visible in my documentation here.

The output from the first code cell in the section should be rendered in red.

In my conf.py, I've set the option nb_render_text_lexer = "myst-ansi".

If I set `nb_render_text_lexer = "none", I can see the ANSI codes in the cell output, so I assume that the library I use for colouring the output (wasabi) works:

[38;5;1mHello world![0m

Link to your repository or website

https://applied-language-technology.mooc.fi/html/notebooks/part_iii/03_pattern_matching.html#examining-matches-in-context-using-concordances

Steps to reproduce

I build the documentation locally, but my conf.py is available here.

The version of Python you're using

Python 3.9.9

Your operating system

macOS 10.14.6.

Versions of your packages

Jupyter Book      : 0.12.1
External ToC      : 0.2.3
MyST-Parser       : 0.15.2
MyST-NB           : 0.13.1
Sphinx Book Theme : 0.1.7
Jupyter-Cache     : 0.4.3
NbClient          : 0.5.9

Additional context

Here's the configuration of the MyST parser when building the documentation:

myst v0.15.2: MdParserConfig(renderer='sphinx', commonmark_only=False, enable_extensions=['dollarmath'], dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], url_schemes=['http', 'https', 'mailto', 'ftp'], heading_anchors=None, heading_slug_func=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'], words_per_minute=200)

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[chrisjsewell]

Heya, your example output does not contain the \x1b ANSI escape sequences expected by myst-ansi. For example,

In [1]: import pygments
In [2]: lexer = pygments.lexers.get_lexer_by_name("myst-ansi")
In [2]: list(lexer.get_tokens("[38;5;1mHello world![0m"))
Out[2]: [(Token.Text, '[38;5;1mHello world![0m\n')]
In [3]: list(lexer.get_tokens("\x1b[38;5;1mHello world!\x1b[0m"))
Out[3]: [(Token.Color.C1, 'Hello world!'), (Token.Text, '\n')]

I would ask the wasabi guys if they know of a suitable existing https://pygments.org lexer to complement their package.

[chrisjsewell]

You can also see these are required directly in your terminal:

[thiippal]

Hi @chrisjsewell!

Thanks for the prompt reply!

I looked at the source of the HTML document rendered using the option nb_render_text_lexer = "none" and I'm not exactly sure if the escape sequences are there, but there is a single strange symbol preceding the colour information on my Mac:

<div class="output stream highlight-none notranslate"><div class="highlight"><pre><span></span>Hello world!

When I set nb_render_text_lexer = "myst-ansi", I get the following output in the HTML source:

<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span><span class=" -Color -Color-C1">Hello world!</span>

To me it seems that the colour information is successfully added to the HTML document, but somehow does not show up properly. Then again, I am not really familiar at all with ANSI / pygments / lexing.

[chrisjsewell]

Ah actually, I see what's going on: We add the CSS to support ANSI color classes here: https://github.com/executablebooks/MyST-NB/blob/2baade03313cf52ac7e7d191bed8f7bae73f6728/myst_nb/_static/mystnb.css#L130

But they don't yet include https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit, i.e. we need to add:

div.highlight .-Color[class*=-C<N>] {
  color: <hex>
}
div.highlight .-Color[class*=-BGC<N>] {
  background-color: <hex>
}

For N={0,255}

[chrisjsewell]

Feel free to submit a PR 😬

[thiippal]

Hi @chrisjsewell, I will give this a try!

[thiippal]

Thanks @chrisjsewell for your help; just opened a pull request (#379)!