Add compatibility to the latest versions of myst-nb

[maximlt]

myst-nb has been completely rewritten in 0.14.0. nbsite uses it in the render_notebook function that reads a notebook, parses it and renders it as a docutils document. This code had to be almost entirely changed after myst-nb's new version.

I marked this as WIP because:

• the conda-forge feedstock of myst-nb is still at the 0.13.2 version (https://github.com/conda-forge/myst-nb-feedstock/pull/15). I've tested my changes locally by installing 0.14 with pip but I'd like myst-nb to be available on conda-forge before this is merged.
• myst-nb has released 0.15, from what I've seen there shouldn't be changes required but I haven't tested that yet
• I see a lot of these warning messages /Users/mliquet/work/Temp/dochvplot/hvplot/doc/homepage.rst:190002: WARNING: skipping unknown output mime type: application/vnd.holoviews_exec.v0+json [mystnb.unknown_mime_type] during the build. They don't seem to have any bad consequences though, the HoloViews plots I was building did show up correctly. Yet I'd like to make sure there's nothing wrong, and if not silence these warnings.

@philippjfr I was a little confused by the jupyter_execute folder I saw being created and containing evaluated notebooks when I was running nbsite build multiple times in a row. myst-nb doesn't actually use nbconvert, instead it relies on jupyter-cache that is the package that executes the notebooks for myst-nb and is responsible for creating this folder. The only case where we want to have myst-nb execute notebook is I believe to build Param's docs, which doesn't execute nbsite generate-rst but just copies the notebooks from the /examples folder to /docs. I would suggest that by default we disable the execution of notebooks by myst-nb/jupyter-cache when it finds some in the /docs folder. Param's docs would have to override this setting to enable it. What do you think?

# in conf.py

# myst-nb < 0.14
jupyter_execute_notebooks = "off"
# myst-nv >= 0.14
nb_execution_mode = "off"

[philippjfr]

The only case where we want to have myst-nb execute notebook is I believe to build Param's docs, which doesn't execute nbsite generate-rst but just copies the notebooks from the /examples folder to /docs. I would suggest that by default we disable the execution of notebooks by myst-nb/jupyter-cache when it finds some in the /docs folder. Param's docs would have to override this setting to enable it.

Not sure I entirely follow this part, the way I understood most of our doc builds is that generate-rst will create RST stubs with a title for the notebook and then insert the notebook directive. It will then go off and execute the notebook and copy the evaluated version into the /doc folder to ensure that on subsequent runs the evaluation is cached. As long as this caching still works I have no opinion on how it's achieved.

[maximlt]

Not sure I entirely follow this part, the way I understood most of our doc builds is that generate-rst will create RST stubs with a title for the notebook and then insert the notebook directive. It will then go off and execute the notebook and copy the evaluated version into the /doc folder to ensure that on subsequent runs the evaluation is cached. As long as this caching still works I have no opinion on how it's achieved.

Yes this is the logic implemented by the notebook directive which will skip the notebook evaluation if it finds it sitting next to the .rst stub file. This isn't changed at all by this PR:

https://github.com/pyviz-dev/nbsite/blob/c4f290e4fbc7b2e7d0ec63d648b1f06067ec9ab4/nbsite/nbbuild.py#L291-L293

However as myst-nb is one of the default extensions of nbsite, the second time you run a docs build (locally, this doesn't happen in the CI) then it will also run and cache (in its own cache folder0 the notebooks that have been previously evaluated with the notebook directive.

https://github.com/pyviz-dev/nbsite/blob/c4f290e4fbc7b2e7d0ec63d648b1f06067ec9ab4/nbsite/shared_conf.py#L41-L42

Maybe the cleanest approach is simply to replace the myst_nb extension if the share_conf.py file by the myst_parser one so that only MarkDown files are parsed. Param's doc build, which is slightly different than the other ones, would just have to add myst_nb in its conf.py.

[maximlt]

myst-nb 0.16.0 has been released on conda-forge: https://anaconda.org/conda-forge/myst-nb

[maximlt]

I've adapted the code to myst-nb 0.16 (latest version) as adapting it to 0.14 wasn't enough, there were breaking changes too in the recent versions.

I'm still getting these warnings when myst-nb renders a notebook:

WARNING: skipping unknown output mime type: application/vnd.holoviews_exec.v0+json [mystnb.unknown_mime_type]

I'm not particularly worried about it, I'd just like to understand better what they could imply, to silent them.

[maximlt]

Superseeded by https://github.com/pyviz-dev/nbsite/pull/246