Websocket connection failed as it is closed early when using async JEG docker image in JupyterHub Kubernetes

[kevin-bates]

Hi @jaiminkhanderia - thanks for the issue. In reading the email notification (since removed from the description), I noticed you were using the nb2kg image. If you're using Notebook 6.0 or greater, we strongly recommend using the NB2KG layer that is built directly into Notebook. This makes deployments much simpler as the NB2KG server extension and its configuration is no longer necessary. The gateway is used merely by the presence of --gateway-url=http://gateway-host:port (or its equivalent via config file, etc.).

Also, I'm curious as to the nature of what the issue was and, I'm assuming, how it was resolved (by virtue of this issue being closed) for the benefit of others. Was there an extra configuration step necessary, etc? If you could share that insight, it would be helpful. Thank you.

[jaiminkhanderia]

Hey @kevin-bates, thank you for your response.

Yes, we're using Notebook >= 6.0 and I didn't exactly get the part of using the NB2KG layer that is built directly into Notebook. Are you suggesting that we should directly use notebook server to connect to EG as mentioned here? Could you please elaborate on that? Is there a docker image that we should use directly or any other approach by what you meant?

Regarding the issue, we figured out that it was being caused when we were trying to use a custom image built on nb2kg:2.0.0 where we were installing and enabling the nbextensions for all users in JupyterHub environment by default when their server starts. If we revert back to using nb2kg:2.0.0, we don't see any issue with the async JEG docker image and that's why we closed the issue.

Also, I would like to thank you for releasing the async tag for JEG as we were eagerly waiting for this feature.

[kevin-bates]

Thank you for the update.

Yes, to use EG via NB 6, just take a notebook docker image (with version >= 6) and use it. Where you've configured KG_URL, you'll need to set JUPYTER_GATEWAY_URL instead. You should see applicable "KG_"-related variables and (now) configurable options by running jupyter notebook --help-all and looking for the GatewayClient "section".

Here's that output for reference:

GatewayClient options
---------------------
--GatewayClient.auth_token=<Unicode>
    Default: None
    The authorization token used in the HTTP headers.
    (JUPYTER_GATEWAY_AUTH_TOKEN env var)
--GatewayClient.ca_certs=<Unicode>
    Default: None
    The filename of CA certificates or None to use defaults.
    (JUPYTER_GATEWAY_CA_CERTS env var)
--GatewayClient.client_cert=<Unicode>
    Default: None
    The filename for client SSL certificate, if any.
    (JUPYTER_GATEWAY_CLIENT_CERT env var)
--GatewayClient.client_key=<Unicode>
    Default: None
    The filename for client SSL key, if any.  (JUPYTER_GATEWAY_CLIENT_KEY env
    var)
--GatewayClient.connect_timeout=<Float>
    Default: 40.0
    The time allowed for HTTP connection establishment with the Gateway server.
    (JUPYTER_GATEWAY_CONNECT_TIMEOUT env var)
--GatewayClient.env_whitelist=<Unicode>
    Default: ''
    A comma-separated list of environment variable names that will be included,
    along with their values, in the kernel startup request.  The corresponding
    `env_whitelist` configuration value must also be set on the Gateway server -
    since that configuration value indicates which environmental values to make
    available to the kernel. (JUPYTER_GATEWAY_ENV_WHITELIST env var)
--GatewayClient.headers=<Unicode>
    Default: '{}'
    Additional HTTP headers to pass on the request.  This value will be
    converted to a dict. (JUPYTER_GATEWAY_HEADERS env var)
--GatewayClient.http_pwd=<Unicode>
    Default: None
    The password for HTTP authentication.  (JUPYTER_GATEWAY_HTTP_PWD env var)
--GatewayClient.http_user=<Unicode>
    Default: None
    The username for HTTP authentication. (JUPYTER_GATEWAY_HTTP_USER env var)
--GatewayClient.kernels_endpoint=<Unicode>
    Default: '/api/kernels'
    The gateway API endpoint for accessing kernel resources
    (JUPYTER_GATEWAY_KERNELS_ENDPOINT env var)
--GatewayClient.kernelspecs_endpoint=<Unicode>
    Default: '/api/kernelspecs'
    The gateway API endpoint for accessing kernelspecs
    (JUPYTER_GATEWAY_KERNELSPECS_ENDPOINT env var)
--GatewayClient.kernelspecs_resource_endpoint=<Unicode>
    Default: '/kernelspecs'
    The gateway endpoint for accessing kernelspecs resources
    (JUPYTER_GATEWAY_KERNELSPECS_RESOURCE_ENDPOINT env var)
--GatewayClient.request_timeout=<Float>
    Default: 40.0
    The time allowed for HTTP request completion.
    (JUPYTER_GATEWAY_REQUEST_TIMEOUT env var)
--GatewayClient.url=<Unicode>
    Default: None
    The url of the Kernel or Enterprise Gateway server where kernel
    specifications are defined and kernel management takes place. If defined,
    this Notebook server acts as a proxy for all kernel management and kernel
    specification retrieval.  (JUPYTER_GATEWAY_URL env var)
--GatewayClient.validate_cert=<Bool>
    Default: True
    For HTTPS requests, determines if server's certificate should be validated
    or not. (JUPYTER_GATEWAY_VALIDATE_CERT env var)
--GatewayClient.ws_url=<Unicode>
    Default: None
    The websocket url of the Kernel or Enterprise Gateway server.  If not
    provided, this value will correspond to the value of the Gateway url with
    'ws' in place of 'http'.  (JUPYTER_GATEWAY_WS_URL env var)

[jaiminkhanderia]

Thank you for the explanation @kevin-bates.

We'll try this out and let you know about the same.

Thank you once again for your help!