search

jupyterlite / pyodide-kernel

Python kernel for JupyterLite powered by Pyodide
https://jupyterlite-pyodide-kernel.readthedocs.io/en/latest/_static/
BSD 3-Clause "New" or "Revised" License
46 stars 23 forks source link

Option "Using a custom Pyodide distribution" (specified in `jupyter-lite.json`) not working #59

Closed michaelweinold closed 2 months ago

michaelweinold commented 1 year ago

Description

Even though I specify the most recent version of pyodide==0.24.0 in the jupyter-lite.json file, as per the JupyterLite Documentation: "Using a custom Pyodide distribution":

{
  "jupyter-lite-schema-version": 0,
  "jupyter-config-data": {
    "disabledExtensions": [
      "@jupyterlab/drawio-extension",
      "jupyterlab-kernel-spy",
      "jupyterlab-tour"
    ],
    "@jupyterlite/pyodide-kernel-extension:kernel": {
      "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.24.0/full/pyodide.js"
    },
  }
}

and the site build from the GH actions workflow

- name: Build the JupyterLite site
        run: |
          cp README.md content
          jupyter lite build --contents content --output-dir dist

completes successfully, the pyodide version does not change from the default 0.23.4 shipped with jupyterlite-pyodide-kernel==0.1.1:

A WebAssembly-powered Python kernel backed by Pyodide
import pyodide
pyodide.__version__
'0.23.4'
import pyodide_kernel
pyodide_kernel.__version__
'0.1.1'

Is there another way to specify the exact version of Pyodide during the build?

Reproduce

Specify pyodide=0.24.0 in the

  1. Use the above .json config file in a fork of the JupyterLite demo.

Expected behavior

The most recent version of pyodide is used.

Context

Browser Output 
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.10.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/messaging (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.12.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/coreutils (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.11.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/signaling (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.9.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/algorithm (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/translation (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/settingregistry (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/notebook (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 6.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/services (required ^7.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.8.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/properties (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 No satisfying version (^4.0.5) of shared module @jupyterlab/outputarea found in shared scope default.
Available versions: 3.5.3 from _JUPYTERLAB.CORE_OUTPUT
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/logconsole (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/rendermime (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 1.10.4 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/disposable (required ^2.0.0)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.a37e37c87d212fe85e13.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/mainmenu (required ^4.0.5)
p @ remoteEntry.a37e37c87d212fe85e13.js:1
remoteEntry.b15a25cb741a6c7381f8.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
s @ remoteEntry.b15a25cb741a6c7381f8.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/apputils (required ^4.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 1.9.2 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/algorithm (required ^2.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^2.0.1)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.6a76d3e37f02d3977b44.js:1 Unsatisfied version 3.5.3 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/ui-components (required ^4.0.0)
d @ remoteEntry.6a76d3e37f02d3977b44.js:1
remoteEntry.ade722230aea9b7793b2.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
u @ remoteEntry.ade722230aea9b7793b2.js:1
remoteEntry.edc72ee71d4ec67598f6.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
s @ remoteEntry.edc72ee71d4ec67598f6.js:1
remoteEntry.d50eb71a2b1d4b9ee0b9.js:1 Unsatisfied version 1.37.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @lumino/widgets (required ^1.37.2)
s @ remoteEntry.d50eb71a2b1d4b9ee0b9.js:1
service-manager.ts:50 Registering new JupyterLite ServiceWorker https://michaelweinold.github.io/jl_pyodide_test/service-worker-f7de73d.js
service-manager.ts:52 JupyterLite ServiceWorker was sucessfully registered
index.ts:235 Kernel filesystem and JupyterLite contents will be synced
index.js:51 Pyodide contents will be synced with Jupyter Contents
pyodide.asm.js:9 Loading micropip, packaging
pyodide.asm.js:9 Loaded packaging, micropip
pyodide.asm.js:9 Loading sqlite3
pyodide.asm.js:9 Loaded sqlite3
pyodide.asm.js:9 Loading decorator, jedi, parso, pygments, six
pyodide.asm.js:9 Loaded decorator, six, parso, jedi, pygments
michaelweinold commented 1 year ago

Hey, @jtpio - sorry for the quick ping. I would greatly appreciate it if you could have a quick look. Is this something that needs to be fixed in either the pyodide-kernel or jupyterlite?

I added a package to pyodide back in June, which was only now included in the 0.24.0 release - I'd very much like to demo that at our conference next week. If would be very helpful for me to find out if I will be able to use the latest version of pyodide within a JupyterLite environment by then.

jtpio commented 1 year ago

@michaelweinold normally the config option as specified in the top comment should do the trick.

But we could also update the Pyodide version in this repo and make a new release. Would you like to open a PR? The diff would be similar to https://github.com/jupyterlite/pyodide-kernel/pull/50.

Thanks!

michaelweinold commented 1 year ago

...ok, since the config option seems to be correct as-is, I opened the PR you suggested: https://github.com/jupyterlite/pyodide-kernel/pull/61

jtpio commented 1 year ago

Maybe there is still an issue with the config option that would have to be fixed anyway (would need to check).

bollwyvl commented 1 year ago

There are more than a few breaking changes in the new release. The first one is the renaming of repodata.json to pyodide-lock.json, which we check in a couple places, but there are probably others.

bollwyvl commented 1 year ago

micropip also changed a lot. #62 is open as an alternative, exploring the changes. Mostly seems to work locally, we'll see what CI says.

michaelweinold commented 2 months ago

Maybe there is still an issue with the config option that would have to be fixed anyway (would need to check).

It would seem so, @jtpio - I tried again with:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            },
        },
    },
}

But I still get:

Screenshot 2024-07-30 at 08 34 06

The GH Actions build log also does not tell me which version was ultimately used.

jtpio commented 2 months ago

@michaelweinold could it be because this JSON snippet contains trailing commas ?

I just tried in a clean folder with jupyterlite-core==0.4.0 and jupyterlite-pyodide-kernel==0.4.0 and it seems to be working fine:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            }
        }
    }
}

image

jtpio commented 2 months ago

could it be because this JSON snippet contains trailing commas ?

If that's the case and it fixes your issue, maybe the jupyter lite build process should validate the jupyter-lite.json if it exists, and error out if it can't be parsed correctly.

michaelweinold commented 2 months ago

...it was the trailing commas. Having removed them:

{
    "jupyter-lite-schema-version": 0,
    "jupyter-config-data": {
        "settingsStorageDrivers": ["asyncStorage"],
        "contentsStorageDrivers": ["asyncStorage"],
        "litePluginSettings": {
            "@jupyterlite/pyodide-kernel-extension:kernel": {
                "pyodideUrl": "https://cdn.jsdelivr.net/pyodide/v0.26.2/full/pyodide.js"
            }
        }
    }
}

Gives me the correct Pyodide version:

Screenshot 2024-07-30 at 11 04 38
jtpio commented 2 months ago

Thanks @michaelweinold for checking :+1:

I opened https://github.com/jupyterlite/jupyterlite/issues/1447 to improve this in the jupyter lite build process.