search

executablebooks / MyST-NB

Parse and execute ipynb files in Sphinx
https://myst-nb.readthedocs.io
BSD 3-Clause "New" or "Revised" License
207 stars 84 forks source link

1.0.0: documentation build fails #558

Closed kloczek closed 1 week ago

kloczek commented 11 months ago

Describe the bug

Looks like documentation build still fail.

Reproduce the bug

I'm building documentation without have myst-nb installed. It is possible to do that with below patch

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,6 +1,8 @@
 # Configuration file for the Sphinx documentation builder.
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 import os
+import sys
+sys.path.insert(0, os.path.abspath(".."))

 # -- Project information -----------------------------------------------------

That kind of hack is listed in example sphinx conf.py https://www.sphinx-doc.org/en/master/usage/configuration.html#example-of-configuration-file

With that patch documentation build fails with

+ /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v7.1.2
making output directory... done
CoconutWarning: Failed to update Coconut Jupyter kernel installation; the 'coconut' kernel might not work properly as a result (try again with 'sudo').
Coconut: Installing Jupyter kernels 'coconut_py', 'coconut_py2', 'coconut_py3'...
Coconut: Successfully installed Jupyter kernels: 'coconut_py', 'coconut_py2', 'coconut_py3'
myst v2.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions={'dollarmath', 'colon_fence', 'deflist', 'amsmath', 'html_image'}, disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_transition=True, words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True)
myst-nb v1.0.0rc0: NbParserConfig(custom_formats={'.Rmd': ('jupytext.reads', {'fmt': 'Rmd'}, False)}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, eval_name_regex='^[a-zA-Z_][a-zA-Z0-9_]*$', execution_mode='cache', execution_cache_path='', execution_excludepatterns=(), execution_timeout=60, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/build/sphinx/.jupyter_cache
loading intersphinx inventory from https://docs.python.org/3.8/objects.inv...
loading intersphinx inventory from https://jupyterbook.org/objects.inv...
loading intersphinx inventory from https://myst-parser.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://markdown-it-py.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://nbclient.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://nbformat.readthedocs.io/en/latest/objects.inv...
loading intersphinx inventory from https://www.sphinx-doc.org/en/master/objects.inv...
intersphinx inventory has moved: https://jupyterbook.org/objects.inv -> https://jupyterbook.org/en/stable/objects.inv
building [mo]: targets for 0 po files that are out of date
writing output...
building [man]: all manpages
updating environment: [new config] 22 added, 0 changed, 0 removed
reading sources... [  9%] authoring/custom-formats
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 287, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 310, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 417, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 438, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 494, in read_doc
    publisher.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 234, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 104, in read
    self.parse()
  File "/usr/lib/python3.8/site-packages/docutils/readers/__init__.py", line 76, in parse
    self.parser.parse(self.input, document)
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/myst_nb/sphinx_.py", line 137, in parse
    nb_renderer: NbElementRenderer = load_renderer(renderer_name)(
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/myst_nb/core/render.py", line 862, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default

Exception occurred:
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/myst_nb/core/render.py", line 862, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default
The full traceback has been saved in /tmp/sphinx-err-tz5bt25a.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
[tkloczko@pers-jacek SPECS]$ cat /tmp/sphinx-err-tz5bt25a.log
# Platform:         linux; (Linux-6.4.8-200.fc38.x86_64-x86_64-with-glibc2.34)
# Sphinx version:   7.1.2
# Python version:   3.8.18 (CPython)
# Docutils version: 0.20.1
# Jinja2 version:   3.1.2
# Pygments version: 2.16.1

# Last messages:
#   [new config]
#   22 added, 0 changed, 0 removed

#   reading sources... [  5%]
#   authoring/basics
#

#   reading sources... [  9%]
#   authoring/custom-formats
#

# Loaded extensions:
#   sphinx.ext.mathjax (7.1.2)
#   alabaster (0.7.13)
#   sphinxcontrib.applehelp (1.0.4)
#   sphinxcontrib.devhelp (1.0.2)
#   sphinxcontrib.htmlhelp (2.0.4)
#   sphinxcontrib.serializinghtml (1.1.9)
#   sphinxcontrib.qthelp (1.0.3)
#   myst_nb (1.0.0rc0)
#   sphinx_copybutton (0.5.1)
#   sphinx_book_theme (unknown version)
#   sphinx.ext.intersphinx (7.1.2)
#   sphinx.ext.autodoc.preserve_defaults (7.1.2)
#   sphinx.ext.autodoc.type_comment (7.1.2)
#   sphinx.ext.autodoc.typehints (7.1.2)
#   sphinx.ext.autodoc (7.1.2)
#   sphinx.ext.viewcode (7.1.2)
#   sphinx_design (0.5.0)

# Traceback:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 290, in build_main
    app.build(args.force_all, args.filenames)
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 351, in build
    self.builder.build_update()
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 287, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 310, in build
    updated_docnames = set(self.read())
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 417, in read
    self._read_serial(docnames)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 438, in _read_serial
    self.read_doc(docname)
  File "/usr/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 494, in read_doc
    publisher.publish()
  File "/usr/lib/python3.8/site-packages/docutils/core.py", line 234, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/lib/python3.8/site-packages/sphinx/io.py", line 104, in read
    self.parse()
  File "/usr/lib/python3.8/site-packages/docutils/readers/__init__.py", line 76, in parse
    self.parser.parse(self.input, document)
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/myst_nb/sphinx_.py", line 137, in parse
    nb_renderer: NbElementRenderer = load_renderer(renderer_name)(
  File "/home/tkloczko/rpmbuild/BUILD/MyST-NB-1.0.0/myst_nb/core/render.py", line 862, in load_renderer
    raise EntryPointError(f"No Entry Point found for {RENDER_ENTRY_GROUP}:{name}")
myst_nb.core.render.EntryPointError: No Entry Point found for myst_nb.renderers:default

List your environment

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------- alabaster 0.7.13 anyio 3.7.1 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 asttokens 2.2.1 attrs 23.1.0 Babel 2.13.1 backcall 0.2.0 beautifulsoup4 4.12.2 bleach 6.1.0 build 1.0.0 cffi 1.16.0 charset-normalizer 3.3.1 click 8.1.7 coconut 3.0.1 colorama 0.4.6 comm 0.1.4 contourpy 1.0.7 cppclean 0.13 cPyparsing 2.4.7.2.2.3 cycler 0.12.1 debugpy 1.8.0 decorator 5.1.1 defusedxml 0.7.1 distro 1.8.0 docutils 0.20.1 exceptiongroup 1.1.3 executing 1.2.0 fastjsonschema 2.18.0 fonttools 4.43.1 gitdb 4.0.10 GitPython 3.1.40 gpg 1.22.0 greenlet 2.0.2 html5lib 1.1 idna 3.4 imagesize 1.4.1 importlib-metadata 6.8.0 importlib-resources 6.1.0 iniconfig 2.0.0 installer 0.7.0 ipykernel 6.26.0 ipython 8.12.0 ipywidgets 8.0.3 jedi 0.19.0 Jinja2 3.1.2 jsonschema 4.19.1 jsonschema-specifications 2023.7.1 jupyter-cache 0.5.0 jupyter_client 8.4.0 jupyter_core 5.3.1 jupyter-events 0.8.0 jupyter_server 2.9.1 jupyter-server-mathjax 0.2.6 jupyter_server_terminals 0.4.4 jupyter-sphinx 0.4.0 jupyterlab-pygments 0.1.2 jupyterlab-widgets 3.0.8 jupytext 1.15.2 kiwisolver 1.4.5 markdown-it-py 3.0.0 MarkupSafe 2.1.3 matplotlib 3.6.3 matplotlib-inline 0.1.6 mdit-py-plugins 0.4.0 mdurl 0.1.2 mistune 2.0.5 mpmath 1.3.0 myst-parser 2.0.0 nbclient 0.8.0 nbconvert 7.8.0 nbdime 3.2.1 nbformat 5.9.2 nest-asyncio 1.5.8 numpy 1.24.4 olefile 0.46 overrides 7.4.0 packaging 23.2 pandas 2.0.3 pandocfilters 1.5.0 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 10.1.0 pkgutil_resolve_name 1.3.10 platformdirs 3.11.0 pluggy 1.3.0 ply 3.11 prometheus-client 0.17.1 prompt-toolkit 3.0.39 psutil 5.9.5 ptyprocess 0.7.0 pure-eval 0.2.2 pycparser 2.21 pydata-sphinx-theme 0.12.0 Pygments 2.16.1 pyparsing 3.1.1 pyproject_hooks 1.0.0 pytest 7.4.3 pytest-datadir 1.5.0 pytest_param_files 0.6.0 pytest-regressions 2.5.0 python-dateutil 2.8.2 python-json-logger 2.0.7 pytz 2023.3 PyYAML 6.0.1 pyzmq 25.1.1 referencing 0.30.2 requests 2.31.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rpds-py 0.10.6 ruamel.yaml 0.18.2 ruamel.yaml.clib 0.2.7 Send2Trash 1.8.2 setuptools 68.2.2 six 1.16.0 smmap 5.0.0 sniffio 1.3.0 snowballstemmer 2.2.0 soupsieve 2.5 Sphinx 7.1.2 sphinx-book-theme 0.3.3 sphinx-copybutton 0.5.1 sphinx_design 0.5.0 sphinx-togglebutton 0.3.2 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.4 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.9 SQLAlchemy 2.0.22 stack-data 0.6.3 sympy 1.12 tabulate 0.9.0 terminado 0.17.1 tinycss2 1.2.1 toml 0.10.2 tomli 2.0.1 tornado 6.3.3 traitlets 5.11.2 typing_extensions 4.8.0 urllib3 1.26.18 wcwidth 0.2.8 webencodings 0.5.1 websocket-client 1.6.4 wheel 0.41.2 widgetsnbextension 4.0.9 zipp 3.17.0 ```
kloczek commented 11 months ago

Q: why documentation build is trying to update some modules? 🤔

agoose77 commented 11 months ago

OK, I can see why this would happen. We use entrypoints to facilitate extensibility. These entrypoints require a package to be installed. This isn't something that we can really fix - is there a reason that you can't run sphinx with the package installed?

kloczek commented 11 months ago

Building package I cannot install anything (build is performed from non-root account) and packaging procedure should not be about installing anything. To be honest I don't understand this approach with entry points .. 🤔

agoose77 commented 10 months ago

I don't think there should be anything wrong with requiring a package to be installed in order for its documentation to be built. I am not intimately familiar with OS packaging processes, however. Is it possible for you to create a virtual environment that is used during building?

kloczek commented 10 months ago

Let me show you something

[tkloczko@pers-jacek SPECS]$ grep %sphinx_build_man -l python-*spec| wc -l; ls -1 python-*.spec | wc -l
671
1217

So it means that I have +1.2k rpm packa build procedures in form of rpm spec files and amongst those files +670 is able to produce man page using sphinx. That %sphinx_build_man rpm macro looks like below:

[tkloczko@pers-jacek SPECS]$ rpm -E %sphinx_build_man
\
        PBR_VERSION=%{version} \
        PDM_BUILD_SCM_VERSION=%{version} \
        SETUPTOOLS_SCM_PRETEND_VERSION=%{version} \
        JARACO_PACKAGING_SPHINX_WHEEL=$(ls -1 $PWD/dist/*whl) \
        /usr/bin/sphinx-build -n -T -b man docs build/sphinx/man

Amongst all those python modules literally NONE requires such special approach which is in this case ..

Q: what makes impossible to use only source tree without installing anything to build documentation? Maybe it is kind of only artificial assumption which is easy to remove from documentation build procedure? 🤔

Building documentation out of ONLY source tree has IMO huge advantage because it provides 100% guarantee that documentation will be build out of source tree ano not out of installed module (which may ne not the same version). 😋

bsipocz commented 1 week ago

As it was mentioned above, having a package available is kind of standard for python documentation; many even use the plot or other directives to generate docs outputs using the given library;

and indeed I don't see any unusual in the configsl and starting a build in a clean new environment using the usual/standard tools (e.g using tox with e.g. tox -e docs-update, or RTD) it all works as expected.

So I would suggest closing this issue unless you have a clear path as well as it builds on RTD without problems, so I would suggest closing this issue.

That being said, I also went ahead, uninstalled myst-nb; and run your man page generating command; from the repo directory. Besides a few warnings I didn't run into any issues either.

$ ~/munka/devel/executablebooks/MyST-NB [MAINT_removing_upper_pins L|✚ 1⚑ 1] $ pip uninstall myst-nb
WARNING: Skipping myst-nb as it is not installed.
$ ~/munka/devel/executablebooks/MyST-NB [MAINT_removing_upper_pins L|✚ 1⚑ 1] $ sphinx-build -n -T -b man docs build/sphinx/man
Running Sphinx v8.0.2
loading translations [en]... done
Coconut: Installing Jupyter kernel 'coconut'...
Coconut: Installing Jupyter kernels 'coconut_py', 'coconut_py2', 'coconut_py3'...
Coconut: Successfully installed Jupyter kernels: 'coconut', 'coconut_py', 'coconut_py2', 'coconut_py3'
loading pickled environment... done
myst v4.0.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions={'dollarmath', 'amsmath', 'deflist', 'colon_fence', 'html_image'}, disable_syntax=[], all_links_external=False, links_external_new_tab=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, fence_as_directive=set(), number_code_blocks=[], title_to_header=False, heading_anchors=0, heading_slug_func=None, html_meta={}, footnote_sort=True, footnote_transition=True, words_per_minute=200, substitutions={}, linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', enable_checkboxes=False, suppress_warnings=[], highlight_code_blocks=True)
myst-nb v1.1.1: NbParserConfig(custom_formats={'.Rmd': ('jupytext.reads', {'fmt': 'Rmd'}, False)}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, eval_name_regex='^[a-zA-Z_][a-zA-Z0-9_]*$', execution_mode='cache', execution_cache_path='', execution_excludepatterns=(), execution_timeout=60, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, code_prompt_show='Show code cell {type}', code_prompt_hide='Hide code cell {type}', number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /Users/bsipocz/munka/devel/executablebooks/MyST-NB/build/sphinx/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
writing output... 
building [man]: all manpages
updating environment: 0 added, 0 changed, 0 removed
reading sources... 
looking for now-outdated files... none found
writing... myst-nb.1 { quickstart authoring/index authoring/basics authoring/jupyter-notebooks authoring/text-notebooks authoring/custom-formats computation/index computation/execute computation/coconut-lang render/index render/format_code_cells render/hiding render/interactive render/inline render/glue configuration docutils reference/api reference/changelog reference/contributing } /Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/format_code_cells.md:391: WARNING: No mime type available in priority list for builder 'man' ('custommimetype') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:51: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:51: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:79: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:79: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:84: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:84: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:84: WARNING: No mime type available in priority list for builder 'man' ('text/html') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/docs/render/interactive.md:84: WARNING: No mime type available in priority list for builder 'man' ('application/javascript') [mystnb.mime_priority] [mystnb.mime_priority]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/myst_nb/core/config.py:docstring of myst_nb.core.config.NbParserConfig.get_cell_level_config:1: WARNING: py:class reference target not found: myst_nb.warnings_.MystNBWarnings [ref.class]
/Users/bsipocz/munka/devel/executablebooks/MyST-NB/myst_nb/sphinx_.py:docstring of myst_nb.sphinx_.Parser.env:1: WARNING: py:class reference target not found: SphinxEnvType [ref.class]
done
build succeeded, 11 warnings.

The manual pages are in build/sphinx/man.