Should we depend on the RTC package by default for 4.0.0?

[fcollonval]

Problem

The work to make Real Time Collaboration optional through a package has started.

When discussing where that code should be hosted on GitHub on the weekly call, the question of should it be or not a default dependency was raised. This issue is for discussing that point.

[fcollonval]

Not shipping it by default will

• avoid users to grab specific dependencies unused in non-RTC mode: for example the document backend server or the file id service
• highlight the important change in behavior of the interaction between the document files and JupyterLab - aka if you have a strongly customize content manager, RTC is not as easy as a flag - you need to deal with the file ID service for example.

Shipping it will

• allow to turn that feature easily
• may become mandatory later to address for example the capture of outputs in the server to avoid lost of data when connection is lost

[ellisonbg]

Right now we are focused on 4.0, so I haven't given much though about what we would focus on after the 4.0 launch. At the same time, I think that the output bug (outputs sent while the frontend is disconnected are lost) is one of the most painful parts of Jupyter for users and would love to see us tackle it during the 4.x release cycle. Given that, I am probably in favor of shipping and enabling it as the default for JupyterLab 4.0.

Another reason to do this is that it will ensure that we discover and fix any remaining issues with RTC or the server side storage of files.

[hbcarlos]

In my opinion, we should not depend on the RTC extension by default. JupyterLab is used in various scenarios, and for now, the JupyterLab-RTC extension only covers one scenario; multiple users connecting to one server where everything runs on the same machine, kernels, file system, etc. There could be other use cases like:

• remote file systems
• users working on JupyterLab Desktop that want to share part of their file system with others

All those use cases can be implemented by different extensions, and users can choose what fits best for them.

[bollwyvl]

At a high level, i feel Installing the flagship jupyterlab package should provide, for the 95% use case:

• the best possible single user in-browser experience we can offer
  • this includes the "second notebook tab stays in sync" experience, especially for intentionally-limited UI like notebook
• the best possible installation experience getting it onto a machine
  • official distributions (and all dependencies) should be available for all platforms where supported pythons run
  • this now includes WASM in the browser and other runtimes
  • you don't need access to the internet
  • this probably includes shipping the documentation website... and it's easier to take action based on things you find in the docs
  • avoid historically challenging install/runtime dependencies, like nodejs, pandoc, latex, rust, c/c++ compilers
  • see also the management of source dependencies

What it probably doesn't need is:

• the best possible collaboration experience for a corporate/managed/hub setting
• the best possible authoring experience for extension authors starting/contributing/testing/releasing and maintaining the quality extensions

In the specific case of RTC: as discussed on the most recent JupyterLab call, @fperez called out one of the ages-old issues of multiple browser tabs sharing a document's content. If there is a way, such as a ServiceWorker or postMessage bus to achieve this without installing/running another rust-based server component, then the "second screen" issue would solved up to the "presenter laptop and phone/table" scenario. If that's so, then the ydoc-based server would not need to be a hard dependency, so long as the client can fall back to this lower-entropy use case... but even that reduced capability could land in 4.1, as long as the messaging was clear that it was coming.

If the ydoc-based server is required to support even two browser tabs on the same computer from the same running server process, then it should probably be depended on by default.

As a greedy aside: describing the requisite lightweight "serverless" API would enable other bespoke clients, like JupyterLite, to follow the pattern to be backed by a non-bespoke backend server (e.g. XMPP, dat, etc.) But I digress.

So my ideal experiences might be described like:

pip install notebook
jupyter notebook

You have a working system for localhost:8888, no multi-machine collaboration, but ideally shared content in multiple tabs on the same machine. You don't have jupyterlab.

pip install jupyterlab jupyter-server-local-collaboration
jupyter lab

You have a working local system with a built-in, but cryptographically-secure sharing, even for a single node.

All the files in your root dir are not shared by default to clients you don't own. You don't have to go read an rfc to find out what rtc stands for, and why it's not actually realtime.

conda install -c conda-forge jupyterlab-devtools # brings in notebook and -collaborative
jupyter lab

You have a working labextension build system, full core source maps, and some tools for benchmarking, accessibility, translation, etc.

For pip installs, you still have to have nodejs on $PATH, unless yet another package exists that brings in an icky private nodejs, as some of the build backends do, or a custom v8, etc.

You have local access to cookiecutter-like features, but they are actually revisioned and treated like software, and support finer-grained deltas

[bollwyvl]

As another side note regarding installability, as noted on gitter where the breaking 3.6 release is already affecting downstreams not even being able to build, with attendant mysterious pyproject-build nested tracebacks.

When used as part of a build-system for jupyter labextension install, jupyterlab brings along the entire toolchain, which ends up being hundreds of packages.

In addition to the above, there should probably be an even smaller component with just enough to run jupyter labextension build. In addition to the unavoidable nodejs dependency, this should really only need jupyter_core, jsonschema and one or two other things, and no historically tricky deps like tornado and pyzmq, or new tricky deps like y-*.

[maartenbreddels]

If it weren't for @bollwyvl helping me out, I would have given up adding a bugfix to ipyleaflet (https://github.com/jupyter-widgets/ipyleaflet/pull/1099). Not to complain, just to share real-life feedback, since we probably don't hear 90+% of these friction issues.

[fperez]

BTW @maartenbreddels - that means that ipyleaflet currently can't be installed in a JLab alongside 3.6, correct? (or as of the next ipyleaflet release with that change merged)?

[maartenbreddels]

It can be installed, nothing changes for users, this is only the build requirement. Runtime dependencies are unchanged.

[jtpio]

Since the JupyterLab 4 final release is approaching and JupyterLab is currently in rc1, it looks like the general feeling would be to not depend on jupyter-collaboration in JupyterLab 4 (or at least for now)?

[jtpio]

Closing as JupyterLab 4 does not depend on jupyter-collaboration.