Error when importing pyodide for sphinx documentation build

[timonmerk]

I am following this tutorial (https://panel.holoviz.org/how_to/wasm/sphinx.html), but unfortunately did not manage to build the documentation which show's the panel in sphix due to an import error based on pyodide in nbsite. I included in a rst file the following pyodide block:

.. pyodide::
   import panel
   settings = {"key1" : 5, "key2" : 45}
   settings_panel = panel.pane.JSON(settings, name="JSON")
   settings_panel

and also included the nbsite_pyodide_conf configuration in the sphinx conf.py:

from panel.io.convert import BOKEH_VERSION, MINIMUM_VERSIONS, PY_VERSION
from panel.io.convert import BOKEH_VERSION, MINIMUM_VERSIONS, PY_VERSION
from panel.io.resources import CDN_DIST

def base_version(version: str) -> str:
    """Extract the final release and if available pre-release (alpha, beta,
    release candidate) segments of a PEP440 version, defined with three
    components (major.minor.micro).

    Useful to avoid nbsite/sphinx to display the documentation HTML title
    with a not so informative and rather ugly long version (e.g.
    ``0.13.0a19.post4+g0695e214``). Use it in ``conf.py``::

        version = release = base_version(package.__version__)

    Return the version passed as input if no match is found with the pattern.
    """
    # look at the start for e.g. 0.13.0, 0.13.0rc1, 0.13.0a19, 0.13.0b10
    pattern = r"([\d]+\.[\d]+\.[\d]+(?:a|rc|b)?[\d]*)"
    match = re.match(pattern, version)
    if match:
        return match.group()
    else:
        return version

version = release = base_version(panel.__version__)

if panel.__version__ != version and (PANEL_ROOT / 'dist' / 'wheels').is_dir():
    py_version = panel.__version__.replace("-dirty", "")
    panel_req = f'./wheels/panel-{py_version}-py3-none-any.whl'
    bokeh_req = f'./wheels/bokeh-{BOKEH_VERSION}-py3-none-any.whl'
else:
    panel_req = f'{CDN_DIST}wheels/panel-{PY_VERSION}-py3-none-any.whl'
    bokeh_req = f'{CDN_DIST}wheels/bokeh-{BOKEH_VERSION}-py3-none-any.whl'

def get_requirements():
    with open('pyodide_dependencies.json') as deps:
        dependencies = json.load(deps)
    requirements = {}
    for src, deps in dependencies.items():
        if deps is None:
            continue
        src = src.replace('.ipynb', '').replace('.md', '')
        for name, min_version in MINIMUM_VERSIONS.items():
            if any(name in req for req in deps):
                deps = [f'{name}>={min_version}' if name in req else req for req in deps]
        requirements[src] = deps
    return requirements

nbsite_pyodide_conf = {
    'PYODIDE_URL': 'https://cdn.jsdelivr.net/pyodide/v0.23.1/full/pyodide.js',
    'requirements': [bokeh_req, panel_req, 'pyodide-http'],
    'requires': get_requirements()
}

When trying to run the documentation with make html I get however this error:

Exception occurred: File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\nbsite\pyodide__init__.py", line 257, in run current_source = self.state_machine.get_source() TypeError: StateMachine.get_source() missing 1 required positional argument: 'line_offset'

I checked the official site of pyodide but could not see that the same line is being called.

For compleness here the full traceback from sphinx:

# Sphinx version: 5.3.0
# Python version: 3.10.12 (CPython)
# Docutils version: 0.18.1 release
# Jinja2 version: 3.1.2
# Last messages:
#   reading sources... [ 66%] nm_projection
#   reading sources... [ 70%] nm_rereference
#   reading sources... [ 73%] nm_resample
#   reading sources... [ 76%] nm_run_analysis
#   reading sources... [ 80%] nm_settings
#   reading sources... [ 83%] nm_sharpwaves
#   reading sources... [ 86%] outline_analysis
#   reading sources... [ 90%] outline_featureestimation
#   reading sources... [ 93%] outline_parametrization
#   reading sources... [ 96%] param
# Loaded extensions:
#   sphinx.ext.mathjax (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\mathjax.py
#   sphinxcontrib.applehelp (1.0.4) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinxcontrib\applehelp\__init__.py
#   sphinxcontrib.devhelp (1.0.2) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinxcontrib\devhelp\__init__.py
#   sphinxcontrib.htmlhelp (2.0.1) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinxcontrib\htmlhelp\__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinxcontrib\serializinghtml\__init__.py
#   sphinxcontrib.qthelp (1.0.3) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinxcontrib\qthelp\__init__.py
#   alabaster (0.7.13) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\alabaster\__init__.py
#   sphinx.ext.autodoc.preserve_defaults (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autodoc\preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autodoc\type_comment.py
#   sphinx.ext.autodoc.typehints (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autodoc\typehints.py
#   sphinx.ext.autodoc (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autodoc\__init__.py
#   sphinx.ext.autosummary (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autosummary\__init__.py
#   sphinx.ext.doctest (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\doctest.py
#   sphinx.ext.intersphinx (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\intersphinx.py
#   sphinx.ext.viewcode (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\viewcode.py
#   numpydoc (1.5.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\numpydoc\__init__.py
#   sphinx.ext.napoleon (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\napoleon\__init__.py
#   sphinx.ext.autosectionlabel (5.3.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\ext\autosectionlabel.py
#   sphinx_gallery.gen_gallery (0.13.0) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx_gallery\gen_gallery.py
#   nbsphinx (0.9.2) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\nbsphinx\__init__.py
#   recommonmark (0.7.1) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\recommonmark\__init__.py
#   nbsite.pyodide (0.5) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\nbsite\pyodide\__init__.py
#   pydata_sphinx_theme (unknown version) from C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\pydata_sphinx_theme\__init__.py
Traceback (most recent call last):
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\cmd\build.py", line 281, in build_main
    app.build(args.force_all, args.filenames)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\application.py", line 347, in build
    self.builder.build_update()
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\builders\__init__.py", line 310, in build_update
    self.build(to_build,
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\builders\__init__.py", line 326, in build
    updated_docnames = set(self.read())
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\builders\__init__.py", line 433, in read
    self._read_serial(docnames)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\builders\__init__.py", line 454, in _read_serial
    self.read_doc(docname)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\builders\__init__.py", line 510, in read_doc
    publisher.publish()
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\io.py", line 104, in read
    self.parse()
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\readers\__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\sphinx\parsers.py", line 78, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 2779, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\statemachine.py", line 240, in run
    context, next_state, result = self.check_line(
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\statemachine.py", line 452, in check_line
    return method(match, context, next_state)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 2352, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 2364, in explicit_construct
    return method(self, expmatch)
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 2101, in directive
    return self.run_directive(
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\docutils\parsers\rst\states.py", line 2151, in run_directive
    result = directive_instance.run()
  File "C:\Users\ICN_admin\Anaconda3\envs\pynm-test\lib\site-packages\nbsite\pyodide\__init__.py", line 257, in run
    current_source = self.state_machine.get_source()
TypeError: StateMachine.get_source() missing 1 required positional argument: 'line_offset'

[iamsiddhantsahu]

@timonmerk Where you able to resolve this error? I am also getting the same error.

[m3lab-zzl]

Same error. Though it's possible to manually add line_offset to get_source() in the pyodide/__init__.py, and the output html does have a green play button to 'launch' pyodide env, but it will fail due to pure python3 tornado not found.

Below are the reproduce steps:

# create mamba env and install packages
mamba create -n test panel -y -c conda-forge -C
mamba activate test
mamba install -c pyviz nbsite -C -y

# init sphinx proj
sphinx-quickstart # init proj
sphinx-build source output # check if this work

# follow the instruction from https://panel.holoviz.org/how_to/wasm/sphinx.html, add content to source/index.rst and source/conf.py, then
sphinx-build source output # should fail for TypeError: StateMachine.get_source() missing 1 required positional argument: 'line_offset'
# open that site-packages/nbsite/pyodide/__init__.py and add a number, such as 4 to get_source() as its argument, then
sphinx-build source output # check if this work

python -m http.server 8000 # open browser and visit localhost:8000

Here shows the green button, click it, and it starts working until stuck at loading panel, open Inspect option from browser, shows

Visit pyodide and tornado site, it seems that some build steps are required. Not sure how panel docs made it, because I also tried MyST markdown format (which is the format of panel doc source code) and build the doc both from Windows10 and WSL2, but they all failed.

I hope there is any progress for the bug fix. @MarcSkovMadsen