jupyter-server / enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
https://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Other
623 stars 222 forks source link

KeyError: 'jinja2_env' if kernel does not exist #797

Closed bsdz closed 4 years ago

bsdz commented 4 years ago

Description

When client requests missing kernel from enterprise gateway, the gateway raise a 404 error that relies on notebook error handling routine which uses jinja2 templates. However, jinja is not set up for enterprise gateway.

Steps to replicate

  1. Used the following vscode launch.json config:

    {
    "version": "0.2.0",
    "configurations": [
        {
            "name": "EntGW",
            "type": "python",
            "request": "launch",
            "pythonPath": "/pyenvs/jupyter/.venv/bin/python",
            "module": "enterprise_gateway",
            "args": [
                "--ip=0.0.0.0",
                "--port=8887",
                "--port_retries=0",
                "--log-level=DEBUG",
                "--EnterpriseGatewayApp.authorized_users=['guest']",
                "--EnterpriseGatewayApp.list_kernels=True",
                "--JupyterWebsocketPersonality.list_kernels=True",
            ],
            "justMyCode": false,
        },
        {
            "name": "JupyterLab",
            "type": "python",
            "request": "launch",
            "pythonPath": "/pyenvs/jupyter/.venv/bin/python",
            "module": "jupyterlab",
            "args": [
                "--gateway-url=http://192.168.1.100:8887",
                "--GatewayClient.http_user=guest",
                "--GatewayClient.http_pwd=guest-password",
                "--log-level=DEBUG",
            ],
            "justMyCode": false,
        }
    ]
    }
  2. Go to runtime dir and remove any kernel/nbserver files.

    $ cd $(jupyter --runtime-dir)
    $ rm nbserver-* kernel-*
  3. Start Enterprise GW using launch config above.

  4. Start Notebook using launch config above.

  5. Create a new ipynb notebook in browser.

  6. Stop Enterprise GW server.

  7. In runtime dir, remove any kernel/nbserver files to simulate missing kernel.

    $ rm nbserver-* kernel-* # simulate missing kernel
  8. Start Enterprise GW again.

The following exception occurs:

[I 2020-03-28 11:26:48.597 EnterpriseGatewayApp] Jupyter Enterprise Gateway 2.1.0 is available at http://0.0.0.0:8887
[D 2020-03-28 11:26:48.601 EnterpriseGatewayApp] Initializing websocket connection /api/kernels/40e5c948-2367-4c07-979c-63051f7a1be7/channels
[W 2020-03-28 11:26:48.603 EnterpriseGatewayApp] No session ID specified
[W 200328 11:26:48 web:1786] 404 GET /api/kernels/40e5c948-2367-4c07-979c-63051f7a1be7/channels (192.168.251.219): Kernel does not exist: 40e5c948-2367-4c07-979c-63051f7a1be7
[D 2020-03-28 11:26:48.610 EnterpriseGatewayApp] Using contents: services/contents
[E 200328 11:26:56 web:1220] Uncaught exception in write_error
    Traceback (most recent call last):
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/web.py", line 1703, in _execute
        result = await result
      File "/home/blair/projects/dev-settings/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/gen.py", line 742, in run
        yielded = self.gen.throw(*exc_info)  # type: ignore
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/services/kernels/handlers.py", line 239, in get
        yield super(ZMQChannelsHandler, self).get(kernel_id=kernel_id)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
        value = future.result()
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/gen.py", line 742, in run
        yielded = self.gen.throw(*exc_info)  # type: ignore
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/base/zmqhandlers.py", line 291, in get
        yield maybe_future(res)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
        value = future.result()
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/gen.py", line 748, in run
        yielded = self.gen.send(value)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/services/kernels/handlers.py", line 221, in pre_get
        kernel = self.kernel_manager.get_kernel(self.kernel_id)
      File "//pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/jupyter_client/multikernelmanager.py", line 279, in get_kernel
        self._check_kernel_id(kernel_id)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/services/kernels/kernelmanager.py", line 387, in _check_kernel_id
        raise web.HTTPError(404, u'Kernel does not exist: %s' % kernel_id)
    tornado.web.HTTPError: HTTP 404: Not Found (Kernel does not exist: 40e5c948-2367-4c07-979c-63051f7a1be7)

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/tornado/web.py", line 1218, in send_error
        self.write_error(status_code, **kwargs)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/base/handlers.py", line 576, in write_error
        html = self.render_template('%s.html' % status_code, **ns)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/base/handlers.py", line 506, in render_template
        template = self.get_template(name)
      File "/pyenvs/jupyter-notebook/.venv/lib/python3.7/site-packages/notebook/base/handlers.py", line 502, in get_template
        return self.settings['jinja2_env'].get_template(name)
    KeyError: 'jinja2_env'

Environment

$ jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : not installed
ipython          : 7.13.0
ipykernel        : 5.1.4
jupyter client   : 6.0.0
jupyter lab      : 2.0.1
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.4
traitlets        : 4.3.3

$ jupyter-enterprisegateway --version
2.1.0

Happy to provide more information if needed.

kevin-bates commented 4 years ago

Thank you for opening this issue and providing the information. Would you like to contribute the required changes for this? We hope to get a release out in the next couple of weeks, and it would be great if it includes this fix.