`--Exporter.preprocessors` for markdown export

[levi-ee]

Hello,

Due to #1300 I'm trying to use --Exporter.preprocessors='["nbconvert.preprocessors.ExecutePreprocessor","nbconvert.preprocessors.TagRemovePreprocessor"]'

While it works with --to notebook I have the following error with --to markdown:

$ jupyter nbconvert sinus_generation.ipynb \
  --Exporter.preprocessors='["nbconvert.preprocessors.ExecutePreprocessor"]' \
  --to markdown --debug

...

[NbConvertApp] Attempting to load template markdown.tpl
[NbConvertApp]     template_path: .
Traceback (most recent call last):
  File "/usr/bin/jupyter-nbconvert", line 11, in <module>
    load_entry_point('nbconvert==5.6.1', 'console_scripts', 'jupyter-nbconvert')()
  File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 270, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 664, in launch_instance
    app.start()
  File "/usr/lib/python3/dist-packages/nbconvert/nbconvertapp.py", line 340, in start
    self.convert_notebooks()
  File "/usr/lib/python3/dist-packages/nbconvert/nbconvertapp.py", line 510, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "/usr/lib/python3/dist-packages/nbconvert/nbconvertapp.py", line 481, in convert_single_notebook
    output, resources = self.export_single_notebook(notebook_filename, resources, input_buffer=input_buffer)
  File "/usr/lib/python3/dist-packages/nbconvert/nbconvertapp.py", line 410, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/exporter.py", line 179, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/exporter.py", line 197, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/templateexporter.py", line 322, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/usr/lib/python3/dist-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/markdown.tpl", line 1, in <module>
    {% extends 'display_priority.tpl' %}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/display_priority.tpl", line 1, in <module>
    {%- extends 'null.tpl' -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 26, in <module>
    {%- block body -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 28, in <module>
    {%- block any_cell scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 31, in <module>
    {%- block codecell scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 41, in <module>
    {%- block output_group -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 45, in <module>
    {%- block outputs scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 47, in <module>
    {%- block output scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 61, in <module>
    {%- block display_data scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/null.tpl", line 62, in <module>
    {%- block data_priority scoped -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/markdown.tpl", line 32, in <module>
    {{ super() }}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/skeleton/display_priority.tpl", line 15, in <module>
    {%- block data_png -%}
  File "/usr/lib/python3/dist-packages/nbconvert/exporters/../templates/markdown.tpl", line 45, in <module>
    ![png]({{ output.metadata.filenames['image/png'] | path2url }})
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 411, in getitem
    return obj[argument]
jinja2.exceptions.UndefinedError: 'nbformat.notebooknode.NotebookNode object' has no attribute 'filenames'

I guess this is normal as NotebookNode may be only used for notebook export (?). Is there a way to make the workaround in #1300 work for Markdown ?

[tuncbkose]

I can't reproduce this. Which version of nbconvert are you using, and can you share a notebook you have this issue with?

[levi-ee]

Version: 7.7.1

A notebook with just this python cell is enough:

import matplotlib.pyplot as plt
plt.plot([1,2,3])
plt.show()

I was able to find a way to make it work, but it still not clear to me...

I think that it can be a Windows/WSL issue, but I don't know why it works with --to notebook but not with --to markdown.

Using the following gen_md.sh bash script:

#!/bin/bash
jupyter nbconvert sinus_generation.ipynb \
--Exporter.preprocessors='["nbconvert.preprocessors.ExecutePreprocessor"]' \
--to markdown

If I first open wsl.exe and run the script, it works:

$ ./gen_md.sh
/home/cc/.local/lib/python3.8/site-packages/traitlets/traitlets.py:2930: FutureWarning: --Exporter.preprocessors=["nbconvert.preprocessors.ExecutePreprocessor"] for containers is deprecated in traitlets 5.0. You can pass `--Exporter.preprocessors item` ... multiple times to add items to a list.
  warn(
[NbConvertApp] Converting notebook sinus_generation.ipynb to markdown
[NbConvertApp] Support files will be in sinus_generation_files/
[NbConvertApp] Writing 26039 bytes to sinus_generation.md

But if I run wsl.exe "./gen_md.sh" from a Windows CMD it fails:

C:\dev\issue_nbconvert>wsl.exe "./gen_md.sh"
[NbConvertApp] Converting notebook sinus_generation.ipynb to markdown
[NbConvertApp] Executing notebook with kernel: python3
Traceback (most recent call last):

...

jinja2.exceptions.UndefinedError: 'nbformat.notebooknode.NotebookNode object' has no attribute 'filenames'

To fix that, I simply created an equivalent batch script when I'm running Windows, and it works.

[tuncbkose]

A notebook with just this python cell is enough:

That is exactly what I tried as well, so I'm not really sure what is going on.