blink1073 commented 4 years ago

Look into the logic for nbextensions and come up with a game plan for the new labextensions

Specifically,

What is the discovery mechanism?
What is the command line interface?
What does the REST API look like (notionally /lab/ext/<foo>)?

Create as a server extension here and then port to jupyterlab.

blink1073 commented 4 years ago

nbextensions are documented here

blink1073 commented 4 years ago

Example:

$ jupyter nbextension list
Known nbextensions:
  config dir: /Users/stslve/miniconda/envs/jlab-pip-2.0/etc/jupyter/nbconfig
    notebook section
      nbdime/index  enabled
      - Validating: OK

blink1073 commented 4 years ago

$ jupyter nbextension help
Work with Jupyter notebook extensions

Subcommands
-----------

Subcommands are launched as `jupyter nbextension cmd [args]`. For information on
using subcommand 'cmd', do: `jupyter nbextension cmd -h`.

install
    Install an nbextension
enable
    Enable an nbextension
disable
    Disable an nbextension
uninstall
    Uninstall an nbextension
list
    List nbextensions

Options
-------

Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.

--debug
    set log level to logging.DEBUG (maximize logging output)
--user
    Apply the operation only for the given user
--system
    Apply the operation system-wide
--sys-prefix
    Use sys.prefix as the prefix for installing nbextensions (for environments, packaging)
--py
    Install from a Python package
--python
    Install from a Python package
--log-level=<Enum> (Application.log_level)
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--config=<Unicode> (JupyterApp.config_file)
    Default: ''
    Full path of a config file.

To see all available configurables, use `--help-all`

Examples
--------

    jupyter nbextension list                          # list all configured nbextensions
    jupyter nbextension install --py <packagename>    # install an nbextension from a Python package
    jupyter nbextension enable --py <packagename>     # enable all nbextensions in a Python package
    jupyter nbextension disable --py <packagename>    # disable all nbextensions in a Python package
    jupyter nbextension uninstall --py <packagename>  # uninstall an nbextension in a Python package

$ jupyter nbextension install help
Install Jupyter notebook extensions

Usage

    jupyter nbextension install path|url [--user|--sys-prefix]

This copies a file or a folder into the Jupyter nbextensions directory. If a URL
is given, it will be downloaded. If an archive is given, it will be extracted
into nbextensions. If the requested files are already up to date, no action is
taken unless --overwrite is specified.

Options
-------

Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.

--debug
    set log level to logging.DEBUG (maximize logging output)
--user
    Apply the operation only for the given user
--system
    Apply the operation system-wide
--sys-prefix
    Use sys.prefix as the prefix for installing nbextensions (for environments, packaging)
--py
    Install from a Python package
--python
    Install from a Python package
--overwrite
    Force overwrite of existing files
--symlink
    Create symlink instead of copying files
-s
    Create symlink instead of copying files
--log-level=<Enum> (Application.log_level)
    Default: 30
    Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
    Set the log level by value or name.
--config=<Unicode> (JupyterApp.config_file)
    Default: ''
    Full path of a config file.
--prefix=<Unicode> (InstallNBExtensionApp.prefix)
    Default: ''
    Installation prefix
--nbextensions=<Unicode> (InstallNBExtensionApp.nbextensions_dir)
    Default: ''
    Full path to nbextensions dir (probably use prefix or user)
--destination=<Unicode> (InstallNBExtensionApp.destination)
    Default: ''
    Destination for the copy or symlink

To see all available configurables, use `--help-all`

Examples
--------

    jupyter nbextension install /path/to/myextension

blink1073 commented 4 years ago

In classic notebook, extensions are JS modules that end up in share/jupyter/nbextensions and are configured in etc/jupyter/nbconfig/notebook.d/. They can be manually installed from a url, a symlink, or a python package. I think we should stick with only Python packages that are enabled, similar to how server extensions work.

blink1073 commented 4 years ago

The proposed layout is share/jupyter/lab/modules/<foo> for the module bundles and etc/jupyter/jupyter_lab_config.d/<foo>.json for config.

blink1073 commented 4 years ago

Hmm, does install even make sense then, jupyter serverextension only has list, enable, and disable

blink1073 commented 4 years ago

Dev workflow assuming no labextension install:

Extension author runs pip install -e . for their extension foo created from our cookiecutter, that uses jupyter_packaging. This builds the extension if needed, copies data files to share/jupyter/lab/modules/foo and creates config file etc/share/jupyter/jupyter_lab_config.d/foo.json
Extension has metadata about which folder has their static files
Extension author runs jupyter lab watch
Extension author makes changes to source files, which are copied to share/jupyter/lab/modules/foo
Extension author refreshes the page to pick up changes from their module

blink1073 commented 4 years ago

Here's how it is used in the notebook server: https://github.com/jupyter/notebook/blob/0df10dee3ee7e8c0f56f879b69d37070fccca1c7/notebook/notebookapp.py#L362

blink1073 commented 4 years ago

Proposed layout:

etc/jupyter/labextensions.d/
share/jupyter/labextensions/foo/schemas
                                                    /themes
                                                   /static
share/jupyter/lab/settings/page_config.json
                                          /overrides.json

TODO for building a compiled version
<fill in static, staging, schema, etc. (maybe settings/build_config.json)

TODO later, make this layer-able ~/.jupyter/lab/user-settings ~/.jupyter/lab/workspaces

jasongrout commented 4 years ago

Notes from our conversation today:

How will directories work?

JUPYTERLAB_CORE_DIR default is /conda/env/share/jupyter/lab/static core_mode is /site_packages/jupyterlab/static sys admin sets to /opt/mylab/jupyter/static

application directory is prefix/share/jupyter/lab

The question is: do we load dynamic extensions on top of this?

--core-mode: loads site-package distributed bundle --no-dynamic-extensions: do not load any dynamic extensions

Current layer stays the same

Except now we build module federation capable bundles, and labextension list lists the dynamic modules

jupyter labextension install: installs an npm package as a package in the application dir bundle

jupyter labextension list: lists all extensions, including dynamic extensions, and indicates when one overrides another.

jupyter lab build: builds the application dir bundle

jupyter lab clean: cleans the application dir bundle

These directories also stay the same:

share/jupyter/lab/settings/page_config.json
                           /overrides.json

Dynamic packages

Distributed in any way, i.e., conda package, pip package, debian package, etc.

Webpack module federation bundle is put into prefix/share/jupyter/labextensions/<MODULE>/

Inside this directory, we have share/jupyter/labextensions/<MODULE>

/schemas -settings schemas
/themes - css
/static - the module federation bundle, including a remoteEntrydle, including a remoteEntry.js

Follow the Jupyter path layering, where more plugins in more specific layers override more general layers.

Enabling and disabling extensions works exactly like in server extensions, i.e., conf.d files in etc/jupyter/labextensions.d/. In general, dynamic extensions override the base application bundled extensions.

index.js

  if ext_disabled:
      do nothing
  elif dynamic_version_available:
      pull in remote
  else:
      pull in local

Templated into page config:

List of dynamic extensions installed (server will proxy the URLs to the right directory)
enabled/disabled extension list
the current stuff as well

Supporting extensions depending on other extensions:

Dynamic extensions get this automatically by depending on the relevant other python packages
We could implement extension dependency tracing for npm extensions, i.e., see if any of your dependencies are extensions, load them, recurse.

blink1073 commented 4 years ago

The rest endpoint should be /labextensions/<foo> to mirror nextensions

jupyterlab / jupyterlab-module-federation

Investigate How to Handle Layered Extensions #3

How will directories work?

Current layer stays the same

Dynamic packages

Templated into page config:

Supporting extensions depending on other extensions: