Formatting issues with nbsphinx

[Sillocan]

Hi, I tried adding a jupyter notebook via nbsphinx to my docs using sphinx-immaterial as the theme. Unfortunately the formatting appears to be a bit broken in comparison to other themes such as sphinx-material. For some reason the input goes onto a separate line from the block indicator, and the output ends up trailing off the screen.

I've created a minimal example below and am adding some screenshots with a comparison. I'm not sure if I am doing something wrong or if there is a formatting issue in the theme.

sphinx-immaterial

sphinx-material

Commands I am running:

python -m venv venv
source venv/bin/activate
pip install -r docs/requirements.txt sphinx-autobuild
sphinx-autobuild docs _build/html
# For running with sphinx-material, i just replace sphinx-immaterial and comment out the cache dir

File structure

``` README.md docs/ conf.py index.rst requirements.txt cookbooks/ cookbook.ipynb ``` README.md ```md ``` docs/conf.py ```python import os project = "Demo bug" master_doc = "index" sphinx_immaterial_external_resource_cache_dir = os.path.abspath("./.cache/external_resources") extensions = [ "m2r2", "sphinx.ext.autosectionlabel", "sphinx_immaterial", "sphinx_panels", "nbsphinx", "IPython.sphinxext.ipython_console_highlighting", "IPython.sphinxext.ipython_directive", ] html_theme = "sphinx_immaterial" html_theme_options = html_theme_options if "html_theme_options" in globals() else {} html_theme_options = { **html_theme_options, "icon": { "repo": "fontawesome/brands/gitlab", }, "repo_url": "https://foobar", "edit_uri": "edit/main/docs", "repo_name": "foo", "repo_type": "gitlab", "features": [ "navigation.expand", # "navigation.tabs", # "toc.integrate", "navigation.sections", # "navigation.instant", # "header.autohide", "navigation.top", "navigation.tracking", "search.highlight", "search.share", "toc.follow", "toc.sticky", "content.tabs.link", "announce.dismiss", ], "palette": [ { "media": "(prefers-color-scheme: light)", "scheme": "default", "primary": "indigo", "accent": "light-blue", "toggle": { "icon": "material/weather-sunny", "name": "Switch to dark mode", }, }, { "media": "(prefers-color-scheme: dark)", "scheme": "slate", "primary": "blue-grey", "accent": "light-blue", "toggle": { "icon": "material/weather-night", "name": "Switch to light mode", }, }, ], } ``` docs/index.rst ```rst .. mdinclude:: ../README.md .. toctree:: :caption: Cookbook :hidden: Notebook ``` docs/requirements.txt ```requirements sphinx>=4.5 sphinx-immaterial sphinx-panels m2r2 mistune traitlets>5.0 nbsphinx ipython ipykernel # sphinx-material ``` docs/cookbooks/cookbook.ipynb ```ipynb { "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Cookbook" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "echo \"This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going \"" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "print(\"this is python output\")\n", "print(\"This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going \")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.5" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" } } }, "nbformat": 4, "nbformat_minor": 2 } ```

See https://github.com/jbms/sphinx-immaterial/issues/218#issuecomment-1433922153 for a zipped sample project

[2bndy5]

You could've just uploaded the sample project as a zip file. That way I can download it, build it, and examine the HTML output myself.

We removed the nbsphinx (& numpy) stuff (in this repo) that was inherited from sphinx-material because it took an unreasonably long time to build and required some Linux-specific dependencies (IIRC)...

[Sillocan]

Fair point. I didn't even think of that for some reason. I'm attaching a zip. It does require a few odd things like pandoc, so I added a Dockerfile if desired. Should just be able to run docker-compose up.

issue-218-with-docker.zip

[Sillocan]

Update: On our instance of readthedocs, we don't get the text scrolling off the screen. So, that's probably not a big deal. But, unfortunately we do still get the extra line.

[2bndy5]

docker compose up doesn't work for me. I'm not a proficient docker user, but it isn't completely foreign to me...

After installing pandoc (locally in windows in the user space - not global space for all users), I still can't build the sample.

It errors out with a rather vague prompt

``` Running Sphinx v6.1.3 making output directory... done building [mo]: targets for 0 po files that are out of date writing output... building [html]: targets for 2 source files that are out of date updating environment: [new config] 2 added, 0 changed, 0 removed 0.00s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation. 0.00s - Debugger warning: It seems that frozen modules are being used, which may 0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off 0.00s - to python to disable frozen modules. 0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation. Notebook error: CellExecutionError in cookbooks\cookbook.ipynb: ------------------ %%bash echo "This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going " ------------------ --------------------------------------------------------------------------- CalledProcessError Traceback (most recent call last) Cell In[1], line 1 ----> 1 get_ipython().run_cell_magic('bash', '', 'echo "This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going "\n') File ~\Downloads\issue-218-with-docker\.env\Lib\site-packages\IPython\core\interactiveshell.py:2430, in InteractiveShell.run_cell_magic(self, magic_name, line, cell) 2428 with self.builtin_trap: 2429 args = (magic_arg_s, cell) -> 2430 result = fn(*args, **kwargs) 2432 # The code below prevents the output from being displayed 2433 # when using magics with decodator @output_can_be_silenced 2434 # when the last Python token in the expression is a ';'. 2435 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False): File ~\Downloads\issue-218-with-docker\.env\Lib\site-packages\IPython\core\magics\script.py:153, in ScriptMagics._make_script_magic..named_script_magic(line, cell) 151 else: 152 line = script --> 153 return self.shebang(line, cell) File ~\Downloads\issue-218-with-docker\.env\Lib\site-packages\IPython\core\magics\script.py:305, in ScriptMagics.shebang(self, line, cell) 300 if args.raise_error and p.returncode != 0: 301 # If we get here and p.returncode is still None, we must have 302 # killed it but not yet seen its return code. We don't wait for it, 303 # in case it's stuck in uninterruptible sleep. -9 = SIGKILL 304 rc = p.returncode or -9 --> 305 raise CalledProcessError(rc, cell) CalledProcessError: Command 'b'echo "This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going "\n'' returned non-zero exit status 1. CalledProcessError: Command 'b'echo "This is a really long line that just keeps on going because it is a really long line that just keeps on going because it is a really long line that just keeps on going "\n'' returned non-zero exit status 1. You can ignore this error by setting the following in conf.py: nbsphinx_allow_errors = True ```

At first glance, the output in the OP looks like a it might be suffering from the same problem that #46 had (which required a monkeypatch on sphinx to resolve). I just need to inspect the HTML to verify my suspicion. Do you have a link to your RTD project? Maybe I can inspect the HTML from there.

[2bndy5]

I finally got it to build with WSL Ubuntu (had to install pandoc from apt since their release assets didn't include my WSL architecture, amd64)...

Still not sure what exactly is going on, but my suspicion seem to be right on track. The code snippets use an odd structure of <pre> and <code> elements. This isn't well supported in the mkdocs-material CSS (which we inherit with intentionally minimal alterations). Still investigating though.

[Sillocan]

Unfortunately it's a corporate edition. I can most likely dump to html if that would be helpful.

EDIT: Ah, glad to see you got it building.

[2bndy5]

Yeah so the really long text running off the screen is because it is structured as

<pre>super long text.................</pre>

where the theme's CSS expects this to be

<pre><code>super long text.................</code></pre>

But that doesn't explain why the input line numbers/statements appear as separate code blocks. I feel like it is supposed to be treated as a <table>, but something is preventing that...

So far, this is pointing to a required patch in whatever extension is generating these specific HTML segments; it certainly isn't sphinx-immaterial, so I'm guessing it is nbsphinx.

[jbms]

There is a jupyter notebook extension for mkdocs that is specifically tested to work with mkdocs-material. It might be helpful to compare.

https://github.com/danielfrg/mkdocs-jupyter

[2bndy5]

With a little tweaking to the HTML about the execution number (had to remove <code> around [n]:) and some CSS

.nbinput.docutils.container,
.nboutput.docutils.container {
  display: flex;
}

.nbinput > .input_area ,
.nboutput > .output_area {
  overflow: auto;
}

.nboutput > .prompt.empty.docutils.container {
  padding: 0 2.25ex;
}

I was able to achieve this:

The CSS code is an easy fix, but we'll have to monkeypatch the nbsphinx output to remove the <code> tags around the execution numbers. The more I explore this, the more it feels extremely "hacky" (not robust); I'm sure there are other elements produced by nbsphinx that the issue's sample doesn't use...

[2bndy5]

The mkdocs extension doesn't seem to support dark scheme very well. Try it on this demo page

[2bndy5]

Looks like nbsphinx uses a jinja template macro that inject a RST directive which adds the code elements around the "execution count". My first impression is that it isn't an easy monkey patch to produce.

Maybe we could elevate this to nbsphinx with a requested option to only add <pre></pre> tags around execution count instead of <pre><code></code></pre>.

[jbms]

Potentially another option would be to add a CSS rule to make <pre><code></code></pre> work as the extension expects, but not sure how feasible that would be.

[2bndy5]

Seen as how the <pre> vs <pre><code> is similar to how production lists were broken, I'm very open to adjusting the theme CSS. But, both cases seem rather specific, so the adjusted CSS would have to be very selective. Also, I'm not sure if that would allow us repeal the monkeypatch solution for #46 as well.

[2bndy5]

There's also the inherited theme JS that attaches the copy-to-clipboard button on any <pre> element that has a <code> element within.

[2bndy5]

I just re-ran the sample with updates from nbsphinx (v0.9.1) and the only remaining issue is a rogue "copy-to-clipboard" icon injected by the theme JS for the execution count.

pip list output

``` Package Version ----------------------------- ----------- alabaster 0.7.13 appdirs 1.4.4 asttokens 2.2.1 attrs 23.1.0 Babel 2.12.1 backcall 0.2.0 beautifulsoup4 4.12.2 bleach 6.0.0 certifi 2022.12.7 charset-normalizer 3.1.0 colorama 0.4.6 comm 0.1.3 css-html-js-minify 2.5.5 debugpy 1.6.7 decorator 5.1.1 defusedxml 0.7.1 docutils 0.19 entrypoints 0.4 executing 1.2.0 fastjsonschema 2.16.3 idna 3.4 imagesize 1.4.1 ipykernel 6.22.0 ipython 8.13.1 jedi 0.18.2 Jinja2 3.1.2 jsonschema 4.17.3 jupyter_client 8.2.0 jupyter_core 5.3.0 jupyterlab-pygments 0.2.2 lxml 4.9.2 m2r2 0.3.3.post2 MarkupSafe 2.1.2 matplotlib-inline 0.1.6 mistune 0.8.4 nbclient 0.7.4 nbconvert 6.5.4 nbformat 5.8.0 nbsphinx 0.9.1 nest-asyncio 1.5.6 packaging 23.1 pandocfilters 1.5.0 parso 0.8.3 pickleshare 0.7.5 pip 22.3.1 platformdirs 3.5.0 prompt-toolkit 3.0.38 psutil 5.9.5 pure-eval 0.2.2 pydantic 1.10.7 Pygments 2.15.1 pyrsistent 0.19.3 python-dateutil 2.8.2 python-slugify 8.0.1 pywin32 306 pyzmq 25.0.2 requests 2.29.0 setuptools 65.5.0 six 1.16.0 snowballstemmer 2.2.0 soupsieve 2.4.1 Sphinx 7.0.0 sphinx-immaterial 0.11.3 sphinx-material 0.0.35 sphinx-panels 0.4.1 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.1 Unidecode 1.3.6 urllib3 1.26.15 wcwidth 0.2.6 webencodings 0.5.1 ```

So, I think this issue now only needs a special CSS rule to not display the injected copy button for nbsphinx execution count numbers:

.nbinput > .prompt > .highlight .md-clipboard.md-icon {
  display: none;
}

But maybe we need to alter the theme JS. I'm not enthusiastic about this approach because the nbsphinx ext (technically in beta) could change output structure at anytime; this is evident by the fact that the original concerns in the OP have been satisfied in nbsphinx since v0.9.0 release.

[2bndy5]

Just to re-iterate, the real problem here exists in nbsphinx code https://github.com/spatialaudio/nbsphinx/blob/f1d3f728f979d77d0e1d356b3d51a2336a94f6dc/src/nbsphinx/__init__.py#L704-L705

where nodes.literal_block generates a <pre><code>[n]:</code></pre> which triggers our theme JS to inject a copy-to-clipboard button. But a <pre>[n]:</pre> would be more desirable to avoid triggering the theme JS about the copy button.

[2bndy5]

This issue's full resolution is pending spatialaudio/nbsphinx#737.

I also raised some other issues about nbsphinx's CSS colors not compatible with both dark and light schemes (spatialaudio/nbsphinx#734 and spatialaudio/nbsphinx#733).

This issue can be closed now as we shouldn't need to compensate for nbsphinx. Keep in mind that there are other Jupyter notebook extensions for Sphinx, namely the more experimental myst-nb extension. I suspect other formatting issues might arise depending on the doc author's choice of extensions. We'll just have to address them appropriately as feedback is reported.

Please use nbsphinx v0.9.1 or newer with this theme.