Create a custom JupyterLab application from React.js with extension support

[echarles]

Useful resources:

• https://github.com/jupyterlab/jupyterlab/blob/main/examples/app/index.js
• https://github.com/jupyter/notebook/blob/main/app/index.template.js
• https://github.com/jupyterlab/jupyterlab/pull/14536
• https://github.com/datalayer-examples/jupyterlab-extensions-example/blob/main/src/jupyterlab.js
• https://github.com/jupyterlab-contrib/jupyterlab-plugin-graph

[lneves12]

Hi @echarles !

First of all thanks for working on this package! I saw you presenting it in one of the Jupyter Con lightning talks and I was very excited about it!

Is this just to add support to extensions coming from an npm package directly? Is there a way to add an extension if you copy all the code to your repo ?

I might be able to help adding support to this. Any guidelines of where to start? (So I don't start working in a complete different solution of what you were envisioning)

Thanks!

[echarles]

Hi @echarles !

Hi @lneves12

First of all thanks for working on this package! I saw you presenting it in one of the Jupyter Con lightning talks and I was very excited about it!

So you were in the audience? Being again on-site at JupyterCon was a great experience.

Is this just to add support to extensions coming from an npm package directly? Is there a way to add an extension if you copy all the code to your repo ?

The goal of this issue is to be able to automatically create a react.js component based on a list of jupyterlab extension (from core or from any third party). The issue with current situation of this repo is that you are loosing the extension system.

I might be able to help adding support to this. Any guidelines of where to start? (So I don't start working in a complete different solution of what you were envisioning)

I would say those files are ideas to start from (you list the extensions you want to load, then. the question is what the react component would look like?)

• https://github.com/jupyterlab/jupyterlab/blob/main/examples/app/index.js
• https://github.com/jupyter/notebook/blob/main/app/index.template.js

This PR https://github.com/jupyterlab/jupyterlab/pull/14536 is also interesting to look at as it introspects the available extensions and displays them. That would be a good start for a React Component (first list them, then load them).

Thanks!

Thank to you.

[echarles]

This is progressing with the creation of a JupyterLabApp component that support any core and 3rd party extension. This is for example a Notebook created with the component.

[lneves12]

@echarles amazing!

I am giving it a try, but for some reason when I import the JupyterLabApp component it just shows white. I can see it renders some divs at least, but nothing is displayed. I followed the example on: https://github.com/datalayer/jupyter-ui/blob/main/packages/react/src/examples/JupyterLabApp.tsx

I see a bunch of errors like:

index.es6.js:2014 Plugin '@jupyterlab/application-extension:paths' failed to activate.
overrideMethod @ console.js:213
eval @ index.es6.js:2014
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:2015 Error: @jupyterlab/application-extension:paths must be activated in JupyterLab.
    at activate (index.js:814:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
eval @ index.es6.js:2015
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:1973 Error: @jupyterlab/application-extension:status must be activated in JupyterLab.
    at activate (index.js:779:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
resolveOptionalService @ index.es6.js:1973
await in resolveOptionalService (async)
eval @ index.es6.js:1847
activatePlugin @ index.es6.js:1847
resolveOptionalService @ index.es6.js:1970
eval @ index.es6.js:1847
activatePlugin @ index.es6.js:1847
resolveRequiredService @ index.es6.js:1937
eval @ index.es6.js:1845
activatePlugin @ index.es6.js:1845
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:2014 Plugin '@jupyterlab/application-extension:status' failed to activate.
overrideMethod @ console.js:213
eval @ index.es6.js:2014
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:2015 Error: @jupyterlab/application-extension:status must be activated in JupyterLab.
    at activate (index.js:779:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
eval @ index.es6.js:2015
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
2index.es6.js:1973 Error: @jupyterlab/application-extension:status must be activated in JupyterLab.
    at activate (index.js:779:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
resolveOptionalService @ index.es6.js:1973
await in resolveOptionalService (async)
eval @ index.es6.js:1847
activatePlugin @ index.es6.js:1847
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:1973 Error: @jupyterlab/application-extension:info must be activated in JupyterLab.
    at activate (index.js:799:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
resolveOptionalService @ index.es6.js:1973
await in resolveOptionalService (async)
eval @ index.es6.js:1847
activatePlugin @ index.es6.js:1847
resolveOptionalService @ index.es6.js:1970
eval @ index.es6.js:1847
activatePlugin @ index.es6.js:1847
resolveRequiredService @ index.es6.js:1937
eval @ index.es6.js:1845
activatePlugin @ index.es6.js:1845
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:2014 Plugin '@jupyterlab/application-extension:info' failed to activate.
overrideMethod @ console.js:213
eval @ index.es6.js:2014
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:2015 Error: @jupyterlab/application-extension:info must be activated in JupyterLab.
    at activate (index.js:799:1)
    at eval (index.es6.js:1850:1)
overrideMethod @ console.js:213
eval @ index.es6.js:2015
Promise.catch (async)
eval @ index.es6.js:2013
start @ index.es6.js:2012
load @ JupyterLabAppAdapter.js:56
await in load (async)
JupyterLabAppAdapter @ JupyterLabAppAdapter.js:24
eval @ JupyterLabApp.js:19
commitHookEffectListMount @ react-dom.development.js:23150
commitPassiveMountOnFiber @ react-dom.development.js:24926
commitPassiveMountEffects_complete @ react-dom.development.js:24891
commitPassiveMountEffects_begin @ react-dom.development.js:24878
commitPassiveMountEffects @ react-dom.development.js:24866
flushPassiveEffectsImpl @ react-dom.development.js:27039
flushPassiveEffects @ react-dom.development.js:26984
eval @ react-dom.development.js:26769
workLoop @ scheduler.development.js:266
flushWork @ scheduler.development.js:239
performWorkUntilDeadline @ scheduler.development.js:533
Show 1 more frame
Show less
index.es6.js:1973 Error: @jupyterlab/application-extension:info must be activated in JupyterLab.

do you know what I might be doing wrong?

If I use:

        <Jupyter
            jupyterServerHttpUrl="https://oss.datalayer.tech/api/jupyter"
            jupyterServerWsUrl="wss://oss.datalayer.tech/api/jupyter"
            jupyterToken="60c1661cc408f978c309d04157af55c9588ff9557c9380e4fb50785750703da6"
        >
            <Notebook
                path="ipywidgets.ipynb"
                uid="notebook-nextjs-1"
                externalIPyWidgets={[
                ]}
                cellSidebarMargin={120}
                CellSidebar={CellSidebar}
            />
        </Jupyter>

it renders the nobetook, so I guess it's not a problem importing the library (I am using webpack + pnpm). And btw it was much easier to import @datalayer/jupyter-react this time compared to last time, it just worked.

[echarles]

@lneves12 Thx for trying 👍

index.es6.js:2014 Plugin '@jupyterlab/application-extension:paths' failed to activate.

I remember having that error and had to add more information in the index.html https://github.com/datalayer/jupyter-ui/blob/dfdc4ca92da154647227b4f61da6f09b424499a6/packages/react/public/index.html#L13-L24

I am not completely sure it will fix your issue, and maybe you already have that in your html, but worth asking.

it renders the nobetook, so I guess it's not a problem importing the library (I am using webpack + pnpm). And btw it was much easier to import @datalayer/jupyter-react this time compared to last time, it just worked.

That is great news to read it works well for you with pnpm.

[lneves12]

Still the same. I will keep investigating.

One suspicious thing is that I see a network failed request to: https://oss.datalayer.tech/api/jupyter/api/settings (404). There is no settings successful request , I am not sure if that's a blocker to render the page.

[lneves12]

ah wait, it might be a problem in the way I am adding the jupyter-config-data.

The url should be: https://oss.datalayer.tech/api/jupyter/lab/api/settings (missing the /lab/, so probably the missing bit is the appUrl="/lab"

[lneves12]

probably something else. I fixed the settings, but I still have the same errors

[echarles]

Not sure if you can checkout the complete repo, build it and run the example from source? (use JupyterLabApp or JupyterLabHeadlessApp in https://github.com/datalayer/jupyter-ui/blob/76830f6a8ebe400947fffb617827256803e2ee80/packages/react/webpack.config.js#L13-L23)

... just to validate things...

Otherwise, if you can share a public version of your code/repo, I am happy to have a look there and see what is going on.

[lneves12]

Running the example of the repo it works well!!

Anyway, it would still be nice to have it working in the project I was including it in. I will keep troubleshooting it and I will let you know if I find the cause. The component loads fine, since I can see in the network tab that it even initializes the kernel, but the UI just doesn't load. The root error seems to be: '@jupyterlab/application-extension:paths' failed to activate, but it doesn't give many details of why it failed.

It might be related to the fact that I am using pnpm or something missing in my webpack config.

[echarles]

I hit the @jupyterlab/application-extension:paths' failed to activate and from what I remember that was due to missing config in html. Can you confirm you have similar configs in the head of your host HTML? Can you try with yarn 3, jupyterlab requires that (also if you can give access to the code, I can have a look

https://github.com/datalayer/jupyter-ui/blob/dfdc4ca92da154647227b4f61da6f09b424499a6/packages/react/public/index.html#L6-L25

[lneves12]

yeah, I also saw that happening in this example: https://github.com/datalayer/jupyter-ui/tree/main/examples/cra then I added the extra configs to the html file and JupyterLabApp worked.

For some reason I also tried here: https://github.com/datalayer-examples/jupyter-react-webpack-example and adding the extra config properties I still have the same error, so that's probably what's happening to me as well, I tried both yarn 1 (the one installed on my machine) and yarn 3

(datalayer) *[main][~/projects/jupyter-react-webpack-example]$ conda activate datalayer
(datalayer) *[main][~/projects/jupyter-react-webpack-example]$ yarn --version          
3.5.0

Do you remember why jupyter lab requires yarn 3? I can try to have it working for pnpm as well, that would be cool.

[echarles]

I have added the jupyterlab app example to both https://github.com/datalayer/jupyter-ui/tree/main/examples/cra and https://github.com/datalayer-examples/jupyter-react-webpack-example.

You will need Yarn 3 for now (hopefully this will change), and this is what the webpack example is giving (hope this will work for you)

(PS: jupyterlab bakes yarn command in python build machinery, and even ships its own copy of yarn they call jlpm).

[lneves12]

Thanks for the example! @echarles

I confirm that it work the example.

Unfortunately, it would be too hard to stop using pnpm and use yarn3 in this project. I am exploring other options so that I can mix both package managers (single-spa, module federation or iframe), I will let you know if I am successfully with any of those :)

I am still trying to make it work with pnpm though, and if I add in my root package.json this:

{
    "pnpm": {
        "overrides": {
            "@jupyter-widgets/base": "6.0.5",
            "@jupyter-widgets/controls": "5.0.6",
            "@jupyter-widgets/html-manager": "1.0.8",
            "@jupyter-widgets/jupyterlab-manager": "5.0.8",
            "@jupyter-widgets/output": "6.0.5",
            "@jupyterlab/application": "4.0.3",
            "@jupyterlab/apputils": "4.1.3",
            "@jupyterlab/attachments": "4.0.3",
            "@jupyterlab/cells": "4.0.3",
            "@jupyterlab/codeeditor": "4.0.3",
            "@jupyterlab/codemirror": "4.0.3",
            "@jupyterlab/completer": "4.0.3",
            "@jupyterlab/console": "4.0.3",
            "@jupyterlab/coreutils": "6.0.3",
            "@jupyterlab/docmanager": "4.0.3",
            "@jupyterlab/docregistry": "4.0.3",
            "@jupyterlab/documentsearch": "4.0.3",
            "@jupyterlab/filebrowser": "4.0.3",
            "@jupyterlab/fileeditor": "4.0.3",
            "@jupyterlab/inspector": "4.0.3",
            "@jupyterlab/javascript-extension": "4.0.3",
            "@jupyterlab/json-extension": "4.0.3",
            "@jupyterlab/launcher": "4.0.3",
            "@jupyterlab/lsp": "4.0.3",
            "@jupyterlab/mainmenu": "4.0.3",
            "@jupyterlab/markdownviewer": "4.0.3",
            "@jupyterlab/markedparser-extension": "4.0.3",
            "@jupyterlab/mathjax-extension": "4.0.0",
            "@jupyterlab/nbconvert-css": "4.0.3",
            "@jupyterlab/nbformat": "4.0.3",
            "@jupyterlab/notebook": "4.0.3",
            "@jupyterlab/observables": "5.0.3",
            "@jupyterlab/outputarea": "4.0.3",
            "@jupyterlab/rendermime": "4.0.3",
            "@jupyterlab/rendermime-extension": "4.0.3",
            "@jupyterlab/rendermime-interfaces": "3.8.3",
            "@jupyterlab/services": "7.0.3",
            "@jupyterlab/settingregistry": "4.0.3",
            "@jupyterlab/statedb": "4.0.3",
            "@jupyterlab/terminal": "4.0.3",
            "@jupyterlab/theme-dark-extension": "4.0.3",
            "@jupyterlab/theme-light-extension": "4.0.3",
            "@jupyterlab/translation": "4.0.3",
            "@jupyterlab/ui-components": "4.0.3",
            "@jupyter/ydoc": "1.1.1",
            "@lumino/algorithm": "2.0.0",
            "@lumino/application": "2.2.0",
            "@lumino/collections": "2.0.0",
            "@lumino/commands": "2.1.2",
            "@lumino/coreutils": "2.1.1",
            "@lumino/default-theme": "2.1.2",
            "@lumino/disposable": "2.1.1",
            "@lumino/domutils": "2.0.0",
            "@lumino/dragdrop": "2.1.2",
            "@lumino/keyboard": "2.0.0",
            "@lumino/messaging": "2.0.0",
            "@lumino/polling": "2.1.1",
            "@lumino/properties": "2.0.0",
            "@lumino/signaling": "2.1.1",
            "@lumino/virtualdom": "2.0.0",
            "@lumino/widgets": "2.2.0",
            "@rjsf/core": "5.3.0",
            "@rjsf/utils": "5.3.0",
            "@rjsf/validator-ajv6": "5.3.0",
            "@rjsf/validator-ajv8": "5.3.0",
            "@types/react": "18.2.12",
            "@types/react-dom": "18.2.5",
            "@jest/core": "29.4.3",
            "@jest/transform": "29.4.3",
            "jest": "29.4.3",
            "jest-environment-jsdom": "29.4.3",
            "ts-jest": "29.0.5",
            "html-webpack-plugin": "5.3.1",
            "htmlparser2": "8.0.1",
            "react": "18.2.0",
            "react-dom": "18.2.0",
            "react-redux": "8.1.2",
            "redux": "4.1.0",
            "redux-observable": "1.2.0",
            "rxjs": "6.6.0",
            "styled-components": "5.3.10",
            "typescript": "5.0.3",
            "webpack": "5.74.0",
            "webpack-cli": "4.10.0",
            "webpack-dev-server": "4.9.3",
            "y-websocket": "1.4.5",
            "yjs": "13.5.47",
            "zustand": "4.4.1"
        }
    }
}

The jupyterlabapp actually loads and I can run cells!! I have other errors though when loading a theme and a custom extensions:

index.es6.js:2003 TypeError: No provider for: @jupyterlab/apputils:IThemeManager.

TypeError: No provider for: @jupyterlab/application:ILabShell.

Hopefully those will be easier to fix for me :) but I guess it's progress having it loading and being able to run cells

[echarles]

@lneves12 I have tried with pnpm:

• On https://pnpm.io/package_json#resolutions , resolutions are merged with pnpm.overrides, I guess you don't need to add pnpm.overrides.
• I have tried with and with pnpm.overrirides, and face the same issue: the transitive dependencies are not resolve on my env. This may be something specific to me (pnpm 8.9.2 on node 20)?

If you could create a repo example with you code, I could try, and also help with the remaining issue for the theme.

For the theme, this is the code of the example that switches the theme and loads additional extension.

https://github.com/datalayer/jupyter-ui/blob/2d3cf14c6e18acfbdac7dfac04d37489edbdd200/packages/react/src/examples/JupyterLabHeadlessApp.tsx#L123-L140