Marimo behind Jupyter / JupyterHub

[s-celles]

Description

Hello,

Quite similar to #904

This may be really interesting to have Marimo available behind Jupyter / JupyterHub as Pluto.jl is doing in https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-julia-notebook with jupyter-pluto-proxy

Kind regards

Suggested solution

Implements something similar to jupyter-pluto-proxy

Alternative

No response

Additional context

No response

[mscolnick]

In the short term, this will slow down our developer velocity since its another environment combination that will need to be test, supported, maintained.

Unless there is a neat solution to support this without sacrificing developer velocity, I not sure this would be supported (at least in core).

Taking a step back: why do you want to do this?

[s-celles]

I understand perfectly that's an important task to achieve that and your team bandwidth is limited. Such a solution will make it possible to edit python files on a VPS without the need to have Python running on your computer. Moreover it could possibly open the multi user features that some other users are requesting (this part will be even harder as you will have to provide isolated environment for each user)

[mscolnick]

Ah got it. I appreciate you explaining the importance of the feature

[s-celles]

My pleasure!

If you want to try this kind of solution quickly with Pluto... here is a docker-compose.yml file for that purpose

version: '3.3'
services:
  jupyter_scipy_notebook:
    image: quay.io/jupyter/scipy-notebook:2024-02-24
    ports:
      - 10000:8888
    volumes:
      - ./local_dir:/home/jovyan/work
      - ~/data:/home/jovyan/data
      - ./certs/private.key:/etc/ssl/notebook/notebook.key
      - ./certs/certificate.crt:/etc/ssl/notebook/notebook.crt
    command: "start-notebook.py \
      --ServerApp.keyfile=/etc/ssl/notebook/notebook.key \
      --ServerApp.certfile=/etc/ssl/notebook/notebook.crt"
    #restart: unless-stopped
    restart: always

and run it using docker compose up (you probably won't need keyfile and certfile as it's not for production (no SSL)

[dazzag24]

Love the project! Addresses many issues I have with Jupyter.

Another important use-case where having the backend running remotely is in cloud computing contexts where it make sense to keep the compute located close to the data. Both for efficiency and security reasons.

For example we run a JupyterHub setup in AWS that spawns per user pods on demand. Our users then access the Juptyer notebooks in their browser while accessing many Tb of data on S3 / EFS that would be slow and costly to process if the notebook was running locally.

[misolietavec]

All that jupyter-*-proxy packages are based on https://github.com/jupyterhub/jupyter-server-proxy project. For example, there are proxy packages for R Shiny, Theia IDE, RStudio, Pluto.jl... I think, creating jupyter-marimo-proxy will make sense later, when marimo will be able to edit more notebooks simultaneously. For now, I am using jupyterhub on remote VPS for editing helper python files. In jupyter terminal I can run marimo edit.. or marimo run.. commands. We have options for specifying host and port already.

[jyio]

https://github.com/marimo-team/marimo/issues/905#issuecomment-1979646282

Moreover it could possibly open the multi user features that some other users are requesting (this part will be even harder as you will have to provide isolated environment for each user)

This is my use case. JupyterHub gives single-user apps multi-user features and, with dockerspawner, container-level isolation.

https://github.com/marimo-team/marimo/issues/905#issuecomment-1985771733

For now, I am using jupyterhub on remote VPS for editing helper python files. In jupyter terminal I can run marimo edit.. or marimo run.. commands. We have options for specifying host and port already.

Specifically, to run a specific notebook, one might do something like:

$ marimo run path/to/notebook.py --port PORT --base-url /user/USER/proxy/absolute/PORT

... and then navigate to https://jupyterhub/user/USER/proxy/absolute/PORT/ ✔

To simplify this, one could add to the jupyterhub_config.py (I'm using dockerspawner, so this is a jupyterhub_config.py within the single-user container):

import os
c = get_config()
c.ServerProxy.servers = {
        'marimo': {
                'command': ['marimo', 'run', 'path/to/notebook.py', '--port', '{port}', '--base-url', '/user/' + os.environ['JUPYTERHUB_USER'] + '/marimo'],
                'absolute_url': True
        }
}

... and then navigate to https://jupyterhub/user/USER/marimo/ ✔

But now I want to edit a notebook, and this is where I hit a wall...

$ marimo edit path/to/notebook.py --port PORT --base-url /user/USER/proxy/absolute/PORT
Error: No such option: --base-url

$ marimo edit --port PORT --base-url /user/USER/proxy/absolute/PORT
Error: No such option: --base-url

Bummer ❌

Is there any way to give marimo edit a base URL? I believe such a feature would make Marimo more usable with jupyter-server-proxy, and it would even be possible to create a proper jupyter-marimo-proxy.

[mscolnick]

hey @jyio, i think we can support marimo edit at a base url. would you mind filing a separate ticket? I can get to it this week

[jyio]

Thank you very much for considering it, @mscolnick.

Base URL might seem to be a small ask on the surface but it could have far-reaching consequences across the entire app, so I definitely appreciate your expertise. I could keep chipping away at the JupyterHub side of things.

[jyio]

Since @mscolnick implemented marimo edit --base-url in record time (thank you!), let me share my minimal viable solution:

Try this Dockerfile if you just want to see it in action. It creates 3 demo users (demo, demo1, and demo2), and you could login to any of them with the same password Free4all! through the magic of DummyAuthenticator. Each user could start its own independent Marimo instance.

Here's what you'd need to configure an existing installation:

1. @mscolnick's latest patch that adds --base-url support to marimo edit. Because this is hot off the press, you must build Marimo yourself if you want to use it right now, or you could wait until it's pip-installable.
2. I did a thing that encapsulates the jupyter-server-proxy config, so you could simply pip install https://github.com/jyio/jupyter-marimo-proxy/archive/main.zip to make it work (it'll make its way over to PyPI someday). All it does is tell the launcher how to launch marimo edit. Works just like jupyter-rsession-proxy and jupyter-pluto-proxy. This is optional... you could skip my package and configure jupyter-server-proxy directly in your jupyterhub_config.py if you want to DIY.
3. There is no 3.

Caveat emptor: This solution doesn't take advantage of the security token feature. Miscreants on a multi-user JupyterHub could potentially sniff out other users' Marimo ports and interfere with them. This could probably be solved by letting JupyterHub generate/maintain the tokens, but I don't know how to make jupyter-server-proxy rewrite the proxied URLs to include the tokens. This is a well-known issue and the usual mitigation is to isolate your users using something like dockerspawner. I have not tried the portwrap option.

I'd be happy to transfer jupyter-marimo-proxy to marimo-team (so you could maintain quality control, trademarks, etc). I don't feel strongly either way :)

[benz0li]

@mscolnick Could you please publish the VS Code extension to the Open VSX Registry, too?

UPDATE: Already asked for. Cross references:

• https://github.com/marimo-team/marimo/issues/1615
• https://github.com/marimo-team/vscode-marimo/issues/26

So people using b-data's/my JupyterLab docker stacks are able to install with code-server.
ℹ️ See https://github.com/open-vsx/publish-extensions/blob/master/CONTRIBUTING.md and https://github.com/eclipse/openvsx/wiki/Publishing-Extensions.

Thank you.

[jyio]

Now that this base URL update has been released, the minimal demo is almost trivial.

[mscolnick]

@jyio thanks the setup gist!

I am going to close this ticket now - please feel free to re-open with more specific requests about deployment.

(@benz0li still waiting on the Open VSX fix upstream)