Jupyterhub "Connection failed"

[alex4200]

I am creating automated tests to test notebooks in jupyterhub (with python-selenium and py.test) in a rather complex environment (I have no standalone setup to reproduce this error, but I will work on it).

So far all works fine.

EXCEPT when I try to log every request and response through browsermob-proxy. This is a java server that acts as a proxy between the selenium webdriver (e.g. firefox driver) and the webpage in question (e.g. the jupyterhub server).

However, only and only when I use this proxy I get a jupyterhub error: "Connection failed"

What are the possible reasons I see that error? What condition(s) trigger this error message?

Addendum

On the server side I get some warnings in the notebooks server:

[I 2018-08-20 14:17:26.702 305933 kernelmanager:98] Kernel started: d84016fb-4d6e-44c7-b577-7de563a9b5b2
[I 2018-08-20 14:17:28.278 305933 handlers:193] Adapting to protocol v5.1 for kernel d84016fb-4d6e-44c7-b577-7de563a9b5b2
[W 2018-08-20 14:17:28.283 305933 log:47] 400 GET /jupyterhub/user/305933/api/kernels/d84016fb-4d6e-44c7-b577-7de563a9b5b2/channels?session_id=47ADDE787AA14E8381C05FD8490A764C (128.178.97.113) 1371.30ms referer=None
[W 2018-08-20 14:17:29.361 305933 handlers:257] Replacing stale connection: d84016fb-4d6e-44c7-b577-7de563a9b5b2:47ADDE787AA14E8381C05FD8490A764C
[W 2018-08-20 14:17:51.406 305933 handlers:257] Replacing stale connection: d84016fb-4d6e-44c7-b577-7de563a9b5b2:47ADDE787AA14E8381C05FD8490A764C
[W 2018-08-20 14:18:15.412 305933 handlers:257] Replacing stale connection: d84016fb-4d6e-44c7-b577-7de563a9b5b2:47ADDE787AA14E8381C05FD8490A764C

Complete working example

Having running a browsermob-proxy on your localhost on port 8080, the following code snippet can reproduce the problem. It also assumes that you are running jupyter notebook in a folder which contains a notebook named Untitled.ipynb. You need to change the token, though...

import time
from browsermobproxy import Client
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Define the Client to the running browsermob-proxy
proxy = Client("localhost:8080", {})

# Create the firefox webdriver
profile = webdriver.FirefoxProfile()
selenium_proxy = proxy.selenium_proxy()
profile.set_proxy(selenium_proxy)
driver = webdriver.Firefox(firefox_profile=profile)

# Navigate to the jupyter main page, and then to the jupyter notebook
proxy.new_har("jupyter")
driver.get("http://localhost:8888/tree?token=d3c806ff34792029a67a4186b720d7ed43127b1e8c26259d")
what = (By.XPATH, "//a/span[contains(text(),'Untitled.ipynb')]")
elem = WebDriverWait(driver, 5).\
                    until(EC.element_to_be_clickable(what))
elem.click()

time.sleep(50)
print (proxy.har)
driver.quit()

Used packages:

• selenium 3.11.0
• geckodriver 0.19.1
• browsermob-proxy 2.1.4
• jupyter notebook 5.6.0

[minrk]

Jupyter uses websockets to talk to the kernels. Since everything works until the websocket connection, my guess is that browsermob-proxy either doesn't support websockets or isn't configured to support them properly.

[alex4200]

@minrk Yes you are correct. browsermob-proxy indeed does not support websockets. But there is a workaround I described here: Essentially you have to handle all websocket request directly, and only handle the other request through websockets...

Looks like I have forgotten about this ticket. Since there is a workaround I will close this ticket..