Can't finish build

[tpike3]

Description

I am integrating jupyterlite-sphinx into mesa-geo readthedocs but my build keeps failing with the following error on in read the docs (no issues if doing jupyter lite build on my local machine)

Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
--
94 | Handler <function jupyterlite_build at 0x7f477f0ddab0> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source', '--XeusAddon.environment_file', 'environment.yml']' returned non-zero exit status 3.)
95 | [jupyterlite-sphinx] Running JupyterLite build

Reproduce

You can see the build at ----

https://app.readthedocs.org/projects/mesa-geo-test/builds/24708569/

Expected behavior

Mesa-Geo docs render with xeus-python kernel

Context

• JupyterLite version: jupyterlite-xeus>0.1.5
• Operating System and version: "ubuntu-22.04"

Thank you for your time. Attached are my conf.txt readthedocs.txt and environment.txt

I feel like this is going to be something very simple but just cant figure out the right combination

[jtpio]

Thanks @tpike3 for reporting :+1:

It may be related to jupyterlite-xeus. There were a few issues recently, maybe @martinRenou or @DerThorsten may no more about this.

In the meantime, you could try to use a different kernel (for example https://github.com/jupyterlite/pyodide-kernel), to see if the build passes.

[tpike3]

Thanks @jtpio will do.

It may also be the directory the build is in. The way I can replicate that error on my local machine if the output directory (jupyterlite_dir) doesn't match.

[martinRenou]

Your logs don't contain much, it looks like the actual error message is being hidden (we don't know why the jupyterlite build command failed)

[tpike3]

Your logs don't contain much, it looks like the actual error message is being hidden (we don't know why the jupyterlite build command failed) Thanks @martinRenou for the help and interest!

Here is the traceback from the readthedocs build ....

writing additional pages... search done
dumping search index in English (code: en)... done
dumping object inventory... done
Updating searchtools for Read the Docs search... 
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/events.py", line 97, in emit
    results.append(listener.handler(self.app, *args))
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/jupyterlite_sphinx/jupyterlite_sphinx.py", line 707, in jupyterlite_build
    completed_process: CompletedProcess[bytes] = subprocess.run(
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/cmd/build.py", line 337, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/application.py", line 353, in build
    self.events.emit('build-finished', None)
  File "/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/conda/latest/lib/python3.10/site-packages/sphinx/events.py", line 108, in emit
    raise ExtensionError(
sphinx.errors.ExtensionError: Handler <function jupyterlite_build at 0x7effb8c75360> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.)

Extension error (jupyterlite_sphinx.jupyterlite_sphinx):
Handler <function jupyterlite_build at 0x7effb8c75360> for event 'build-finished' threw an exception (exception: Command '['jupyter', 'lite', 'build', '--debug', '--contents', 'tutorials/intro_tutorial.ipynb', '--contents', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source/_contents', '--output-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite', '--apps', 'notebooks', '--apps', 'edit', '--apps', 'lab', '--apps', 'repl', '--apps', 'tree', '--apps', 'consoles', '--lite-dir', '/home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/docs/source']' returned non-zero exit status 3.)
[jupyterlite-sphinx] Running JupyterLite build

Right now I am thinking it is one of 2 things (but could be way off)

1. That it the way my directories are set up as I am using a source directory and a build directory, but even when I change structure to match jupyterlite sphinx its doesnt work and my .jupyterlite.doit.db never ends up where it should be and so never gets cleaned as it should here https://github.com/jupyterlite/jupyterlite-sphinx/blob/d50e2465d5bf67cb03897f68abc9a827658fae69/jupyterlite_sphinx/jupyterlite_sphinx.py#L731

2. Or that need to build an environment in my pyproject.toml like jupyterlite-sphinx does here https://github.com/jupyterlite/jupyterlite-sphinx/blob/d50e2465d5bf67cb03897f68abc9a827658fae69/pyproject.toml#L43C1-L47C60

[tpike3]

Well I should pay closer attention to the documentation I set jupyterlite_silence = False and I am getting good traceback information.

However, it is not immediately obvious to me what the issue is.

RTD build details.txt

[jtpio]

Thanks @tpike3.

These two lines at the top of the logs seem to indicate an issue with jupyterlite-xeus:

ValidationError(ValueError("'emscripten' is not a valid Platform"))
ValidationError(ValueError("'emscripten' is not a valid Platform"))

Which should likely be fixed by https://github.com/conda/conda/pull/13962?

In the meantime, I think the workaround might be to use the latest pre-release of jupyterlite-xeus, and add micromamba to the build environment, similar to https://github.com/voila-dashboards/voici/pull/120 which had the same issue.

[jtpio]

@tpike3 looks like this is indeed an issue with jupyterlite-xeus: https://github.com/jupyterlite/jupyterlite-sphinx/pull/179

@martinRenou @DerThorsten maybe we should document the fix somewhere (for example in the jupyterlite-xeus repo) until it is fixed in conda?

[tpike3]

Thanks @jtpio!!! I am getting closer but now I am getting this error--- error libmamba ZSTD decompression error: Unknown frame descriptor

[tpike3]

Thanks @tpike3 for reporting 👍

It may be related to jupyterlite-xeus. There were a few issues recently, maybe @martinRenou or @DerThorsten may no more about this.

In the meantime, you could try to use a different kernel (for example https://github.com/jupyterlite/pyodide-kernel), to see if the build passes.

@jtpio Apologies should have done this earlier, I did switch to a pyodide-kernel and the build works. Working to see how to pre-install packages now.

[jtpio]

@tpike3 you could also continue using jupyterlite-xeus but update to the latest pre-release (as in https://github.com/jupyterlite/jupyterlite-sphinx/pull/179), which should fix the issue.

[tpike3]

@tpike3 you could also continue using jupyterlite-xeus but update to the latest pre-release (as in #179), which should fix the issue.

@jtpio I still cannot get the xeus-python to build into RTD.

[LiteBuildApp] creating folder /home/docs/checkouts/readthedocs.org/user_builds/mesa-geo-test/checkouts/latest/_readthedocs/html/lite/static

139 | error    libmamba ZSTD decompression error: Unknown frame descriptor

Of course I realized pyodide won't work with piplite as Mesa doesn't have a pure python wheel

[jtpio]

Normally this issue should not be related to Pyodide since you are using xeus-python, and also because this is happening a build time (when the JupyterLite site is built) and not at runtime (when opening JupyterLite in a web browser).

Looks like this could be due to a malformed package on emscripten forge or conda forge?

[martinRenou]

error libmamba ZSTD decompression error: Unknown frame descriptor

I believe this should not cause the build to fail. You probably have another error somewhere.