jinja2.exceptions.TemplateNotFound: toc2

[elgalu]

jinja2.exceptions.TemplateNotFound: toc2 failure after upgrading nbconvert >= 6

How to reproduce

pip install nbformat "nbconvert>=6"

jupyter nbconvert --to html_toc "/opt/conda/lib/python3.7/site-packages/nbformat/tests/test2.ipynb"

Error

[NbConvertApp] WARNING | Config option `kernel_spec_manager_class` not recognized by `NbConvertApp`.
[NbConvertApp] Converting notebook /opt/conda/lib/python3.7/site-packages/nbformat/tests/test2.ipynb to html_toc
Traceback (most recent call last):
  File "/opt/conda/bin/jupyter-nbconvert", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/jupyter_core/application.py", line 270, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 664, in launch_instance
    app.start()
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 350, in start
    self.convert_notebooks()
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 524, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 489, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 418, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 181, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 199, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/html.py", line 119, in from_notebook_node
    return super().from_notebook_node(nb, resources, **kw)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 384, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 148, in template
    self._template_cached = self._load_template()
  File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 355, in _load_template
    return self.environment.get_template(template_file)
  File "/opt/conda/lib/python3.7/site-packages/jinja2/environment.py", line 830, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/opt/conda/lib/python3.7/site-packages/jinja2/environment.py", line 804, in _load_template
    template = self.loader.load(self, name, globals)
  File "/opt/conda/lib/python3.7/site-packages/jinja2/loaders.py", line 408, in load
    raise TemplateNotFound(name)
jinja2.exceptions.TemplateNotFound: toc2

Workaround

pip install "nbconvert<6"

[daliagachc]

is it any roadmap to update to nbconvert>6

[pawlodkowski]

seconded. would love if I could get the table of contents html export to work in nbconvert > 6. spent all day working on trying to create a custom exporter myself, but wasn't able to manage.

[leodrivera]

I've been using this workaround reported by @elgalu, as there are still problems with toc export, even when using the version 5.3, that hasn't been officially published yet.

I've written about the issues I got in the other topic related with TOC https://github.com/ipython-contrib/jupyter_contrib_nbextensions/issues/1568

@deflaux Is there a roadmap for fixing the issue?

[deflaux]

@leodrivera I wouldn't know. I have not contributed to this repository. But whenever it is fixed, I look forward to using it again!! The TOC is so nice to have when the Jupyter notebooks exported to HTML are long.

[leodrivera]

Indeed. As I usually write long notebooks, it makes so much easier for reading it.

@deflaux This extension is so much useful and if we could get working, would be something very useful for the community.

I've looked at the last commits and I am pinging them, so maybe we can try to solve this together: @jfbercher @cailiang9 @aduriseti @juhasch @zthxxx

[cailiang9]

maybe this patch to nbconvert can work:

*** a/nbconvert/exporters/templateexporter.py   2020-11-07 12:54:46.000000000 +0800
--- b/nbconvert/exporters/templateexporter.py   2020-11-08 01:22:31.634876123 +0800
***************
*** 531,555 ****
--- 531,562 ----
              # we include root_dir for when we want to be very explicit, e.g.
              # {% extends 'nbconvert/templates/classic/base.html' %}
              paths.append(root_dir)
              # we include base_dir for when we want to be explicit, but less than root_dir, e.g.
              # {% extends 'classic/base.html' %}
              base_dir = os.path.join(root_dir, 'nbconvert', 'templates')
              paths.append(base_dir)

+             compatibility_dir = os.path.join(root_dir, 'jupyter_contrib_nbextensions', 'templates')
+             paths.append(compatibility_dir)
+             compatibility_dir = os.path.join(root_dir, 'nbconvert', 'templates', 'base')
+             paths.append(compatibility_dir)
+             compatibility_dir = os.path.join(root_dir, 'nbconvert', 'templates', 'classic')
+             paths.append(compatibility_dir)
              compatibility_dir = os.path.join(root_dir, 'nbconvert', 'templates', 'compatibility')
              paths.append(compatibility_dir)

          additional_paths = self.template_data_paths
          for path in additional_paths:
              try:
                  ensure_dir_exists(path, mode=0o700)
              except OSError:
                  pass
+         #print("self.extra_template_basedirs:", self.extra_template_basedirs, self.extra_template_paths + additional_paths + paths)

          return self.extra_template_paths + additional_paths + paths

      @classmethod
      def get_compatibility_base_template_conf(cls, name):
          # Hard-coded base template confs to use for backwards compatibility for 5.x-only templates
          if name == 'display_priority':
              return dict(base_template='base')

[cailiang9]

according to https://nbconvert.readthedocs.io/en/latest/external_exporters.html#registering-a-custom-exporter-as-an-entry-point, you can also try: jupyter nbconvert --to jupyter_contrib_nbextensions.nbconvert_support.toc2.TocExporter notebook.ipynb

[leodrivera]

@cailiang9 Thanks for the answer

Unfortunately, it did not solve for me. However, I could solve part of the problem by changing the files hosted in https://gitcdn.xyz/ (and substituting the path here) to the same files hosted in my personal github (inside github.io) and, after that, the js could be effectively loaded and the page appeared. It seems that the gitcdn site is very slow and unstable. It would be very nice if another repo was created, with the github pages options enabled, so that these files could be hosted into it.

But I still have those two issues:

• The 'None' keeps appearing in the top left hand corner. I noticed that it also appeared when using the command:

  jupyter nbconvert --to html --template full notebook.ipynb and after googling it, I found it: https://github.com/jupyter/nbconvert/issues/1652 It is related with the toc template, probably because of the next issue.

• The toc sidebar disappeared. Although there was a fix for this issue (https://github.com/ipython-contrib/jupyter_contrib_nbextensions/pull/1548), it seems it wasn't solved.

[Sieboldianus]

I got toc2 rendered correctly in a notebook html in nbconvert 6.5.0 and jupyterlab 3.4.4. The key was:

• add a custom template and use custom URLs for toc2 sources (CDN) main.css and toc2.js

  <link rel="stylesheet" type="text/css" href="https://my.cdn.org/toc2/main.css">
  <script src="https://my.cdn.org/toc2/toc2.js"></script>

• add a conf.json referencing the classic template (instead of full.tpl)

  {
  "base_template": "classic",
  "mimetypes": {
  "text/html": true
  },
  "preprocessors": {
  "100-pygments": {
      "type": "nbconvert.preprocessors.CSSHTMLHeaderPreprocessor",
      "enabled": true
  },
  "500-reveal": {
    "type": "nbconvert.exporters.slides._RevealMetadataPreprocessor",
    "enabled": true
  }
  }
  }

• in the custom template, instead of full.tpl, extend from classic/index.html.j2

  {%- extends 'classic/index.html.j2' -%}

• add a block to your template, reversing the nbconvert order of requirejs and jquery - see my report here.

  {%- block html_head_js -%}
  {%- block html_head_js_requirejs -%}
  <script src="{{ resources.require_js_url }}"></script>
  {%- endblock html_head_js_requirejs -%}
  {%- block html_head_js_jquery -%}
  <script src="{{ resources.jquery_url }}"></script>
  {%- endblock html_head_js_jquery -%}
  {%- endblock html_head_js -%}

Here is my full working template, you only need to replace URLs for main.css and toc2.js - I don't want mine to be used elsewhere.

nbconvert.tpl

```js {%- extends 'classic/index.html.j2' -%} {# toc2 template for nbconvert >6.0 - created from files in: https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tree/master/src/jupyter_contrib_nbextensions/templates https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tree/master/src/jupyter_contrib_nbextensions/nbextensions/toc2 https://github.com/ipython-contrib/jupyter_contrib_nbextensions/blob/master/src/jupyter_contrib_nbextensions/templates/toc2.tpl https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tree/master/src/jupyter_contrib_nbextensions/nbextensions/toc2 #} {% block output_group -%} {%- if cell.metadata.hide_output -%} {%- else -%} {{ super() }} {%- endif -%} {% endblock output_group %} {% block output_area_prompt %} {%- if cell.metadata.hide_input or nb.metadata.hide_input -%}

{%- else -%} {{ super() }} {%- endif -%} {% endblock output_area_prompt %} {%- block html_head_js -%} {%- block html_head_js_requirejs -%} {%- endblock html_head_js_requirejs -%} {%- block html_head_js_jquery -%} {%- endblock html_head_js_jquery -%} {%- endblock html_head_js -%} {%- block html_head -%} {{ super() }} {% for css in resources.inliner.css -%} {% endfor %} {% for js in resources.inliner.js -%} {% endfor %} {%- endblock html_head -%} ```

Then export your notebook with:

!jupyter nbconvert --to html_toc \
    --output-dir=./html ./test.ipynb \
    --template=./nbconvert.tpl \
    --ExtractOutputPreprocessor.enabled=False  >&- 2>&- # create single output file and suppress output

(from a notebook cell)

screenshot


[palbarede]

% jupyter notebook --version
6.5.2
% pip list|grep nbconvert
nbconvert                         7.2.9

I still have the problem. I am trying your solution.

In my installation, I find toc2.js and main.js in

/home/alba/.conda/envs/jupyter/lib/python3.10/site-packages/jupyter_contrib_nbextensions/nbextensions/toc2/,

which I replace exactly for 'https://replace.with.cdn/toc2/' in your nbconvert.tpl. (Actually this does not change anything in the following.)

I find /home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/compatibility/full.tpl ,

% cat ~/.conda/envs/jupyter/share/jupyter/nbconvert/templates/compatibility/full.tpl {{ resources.deprecated("This template is deprecated, please use classic/index.html.j2") }} {%- extends 'index.html.j2' -%}

I think it is almost what you have done.

I guess that your text beginning with

{
  "base_template": "classic",

should go in some file named conf.json.

I found

% cat /home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic/conf.json
{
  "base_template": "base",
  "mimetypes": {
    "text/html": true
  },
  "preprocessors": {
    "100-pygments": {
      "type": "nbconvert.preprocessors.CSSHTMLHeaderPreprocessor",
      "enabled": true,
      "style": "default"
    }
  }
}

I replace it by your conf.json, including "base_template": "classic".

I also change the first line of /home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic/ so that

% head index.html.j2      
{%- extends 'classic/index.html.j2' -%}

Then the ouput of your program is:

File "/home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic/index.html.j2", line 2, in top-level template code
    {% from 'mathjax.html.j2' import mathjax %}
jinja2.exceptions.TemplateNotFound: mathjax.html.j2

The problem https://github.com/jupyter/nbconvert/issues/1394 has been solved.

Then

% pwd                       
/home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic
% cp ../base/mathjax.html.j2 .

and following the same principle as before add classic/ before mathjax.html.j2 in index.html.j1.

Similar correction needed for jupyter_widgets.html.j2.

Next error:

  File "/home/alba/.conda/envs/jupyter/lib/python3.10/abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

abc.py is the abstract class Python module.

I put print(cls) at line 119 of abc.py.

File "/home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic/index.html.j2", line 3, in top-level template code
    {% from 'classic/jupyter_widgets.html.j2' import jupyter_widgets %}
  [Previous line repeated 973 more times]
  File "/home/alba/.conda/envs/jupyter/share/jupyter/nbconvert/templates/classic/index.html.j2", line 2, in top-level template code
    {% from 'classic/mathjax.html.j2' import mathjax %}
  File "/home/alba/.conda/envs/jupyter/lib/python3.10/_collections_abc.py", line 666, in __rsub__
    if not isinstance(other, Set):
  File "/home/alba/.conda/envs/jupyter/lib/python3.10/abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

[lukelbd]

Spent a lot of time trying to get table of contents to work and just could not get it done. @Sieboldianus I added your nbconvert.tpl template + conf.json file and successfully used it with --to html_toc --template ./nbconvert.tpl, but I just get None printed at the top of the page. Also made sure to enable table of contents sidebar for the notebook in question by following this stackoverflow answer, and manually enabling in jupyter notebook interface.

This is what nbextensions configurator tab shows, and this is the notebook in question:

Might be some weird install issue with toc2 but not sure. Not sure where to go from here, giving up for now. For reference here is my current environment (also using jinja2 3.0.0):

$ pip list | grep -E 'nbconvert|jinja2|nbextension|jupyter
nbconvert                         7.8.0
jupyter_client                    7.4.9
jupyter-console                   6.6.3
jupyter-contrib-core              0.4.0
jupyter-contrib-nbextensions      0.7.0
jupyter_core                      5.3.0
jupyter-events                    0.7.0
jupyter-highlight-selected-word   0.2.0
jupyter-latex-envs                1.4.6
jupyter-lsp                       2.2.0
jupyter-nbextensions-configurator 0.6.1
jupyter_server                    2.7.3
jupyter_server_fileid             0.9.0
jupyter_server_terminals          0.4.4
jupyter_server_ydoc               0.8.0
jupyter-ydoc                      0.2.4
jupyterlab                        3.6.5
jupyterlab-pygments               0.2.2
jupyterlab_server                 2.25.0

[Sieboldianus]

Yes, this gets more and more complicated. My approach worked until last week, when jupyter_contrib_nbextensions failed to install with conda (see #1650). I could work around this by installation with pip. However, then I got a ModuleNotFoundError: No module named 'notebook.base' error. I could work around this by pinning notebook<7.0.0.

These are my current versions:

notebook 6.5.5
jupyter_server 2.7.3
jupyterlab 4.0.6
nbconvert 7.8.0

Here are some links to get you started:

• https://gitlab.vgiscience.de/lbsn/tools/jupyterlab
  • this is the Jupyter lab container that I use (docker)
  • the jupyter env/server is separated from the working environments; you can see the dependencies for jupyrer in the environment_jupyter.yml (this should be the only relevant file that you need to replicate my setup)
• https://gitlab.hrz.tu-chemnitz.de/tud-ifk/jupyter_base_template/-/tree/main/%7B%7Bcookiecutter.project_slug%7D%7D
  • this is my template for creating new jupyter lab-based projects; have a look at conf.json and nbconvert.tpl, these two files should be the only ones you need (nbconvert.tpl is a custom template I created off the classic-template)
• here is an example html generated with the above (with toc extension)
• this is the command I use (inside the notebook):

  !jupyter nbconvert --to html_toc ./test.ipynb \
      --template=../nbconvert.tpl \
      --ExtractOutputPreprocessor.enabled=False >&- 2>&-

Some hints:

• I don't use jupyter_nbextensions_configurator anymore since this seems not compatible anymore with the latest Jupyter lab
• I don't use the toc extension to display toc inside the notebook; I only need it to render/add a toc to nbconvert-generated html files

[zertrin]

I struggled a lot combining all the knowledge in this thread for my case, but finally stumbled a working solution for my use case (using nbconvert<6).

I'm working in an older conda environment with an older version of nbconvert 5.6.1 (for scientific notebooks developed some years ago), and I managed to circumvent both the missing toc2 template issue and also the current problem whereby raw.gitcdn.xyz is now down.

Here are my steps (hoping this can help someone):

• copy toc2.tpl file from $HOME/mambaforge/envs/my-env-name-redacted/lib/python3.7/site-packages/jupyter_contrib_nbextensions/templates/toc2.tpl to a folder of my choice (let's say $HOME/custom-toc2/)
• modified the URLs in $HOME/custom-toc2/toc2.tpl
  • by changing https://min.gitcdn.xyz/cdn/ipython-contrib/jupyter_contrib_nbextensions/master/src/...
  • to https://cdn.jsdelivr.net/gh/ipython-contrib/jupyter_contrib_nbextensions@0.5.1/src/...
  • for both the main.css and toc2.js imports

Resulting $HOME/custom-toc2/toc2.tpl (click to expand)

```html {%- extends 'nbextensions.tpl' -%} {%- block header -%} {{ super() }} {%- endblock header -%} ```

Then finally, using the following command to convert my notebooks:

jupyter nbconvert --to html_toc --template="$HOME/custom-toc2/toc2.tpl" my-notebook.ipynb