search

jupyter / help

:sparkles: Need some help or have some questions? Please visit our Discourse page.
https://discourse.jupyter.org
291 stars 97 forks source link

jupyterhub [ConfigProxy] error: Uncaught Exception #478

Open joergFLI opened 5 years ago

joergFLI commented 5 years ago

I am trying to run JupyterHub on Centos but get a Uncaught Exeption Error

node --version v6.14.3 npm --version 3.10.10 jupyterhub --version 0.9.4

jupyterhub --config jupyterhub_config.py

[I 2018-12-17 14:50:15.070 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4
[I 2018-12-17 14:50:15.070 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4
[I 2018-12-17 14:50:15.072 JupyterHub app:1016] Loading cookie_secret from /home/smrtanalysis/jupyterhub_cookie_secret
[I 2018-12-17 14:50:15.106 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2018-12-17 14:50:15.108 JupyterHub app:1173] No admin users, admin interface will be unavailable.
[I 2018-12-17 14:50:15.198 JupyterHub app:1855] Hub API listening on http://127.0.0.1:9000/hub/
[W 2018-12-17 14:50:15.200 JupyterHub proxy:472] Found proxy pid file: /home/smrtanalysis/jupyterhub-proxy.pid
[W 2018-12-17 14:50:15.200 JupyterHub proxy:489] Proxy still running at pid=56012
[W 2018-12-17 14:50:16.202 JupyterHub proxy:509] Stopped proxy at pid=56012  
[W 2018-12-17 14:50:16.205 JupyterHub proxy:565] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2018-12-17 14:50:16.205 JupyterHub proxy:567] Starting proxy @ http://172.16.0.20:80/
14:50:16.610 [ConfigProxy] info: Proxying http://172.16.0.20:80 to (no default)
14:50:16.617 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
14:50:16.626 [ConfigProxy] error: Uncaught Exception
[C 2018-12-17 14:50:27.322 JupyterHub app:1867] Failed to start proxy 
    Traceback (most recent call last):
      File "/usr/lib/python3.5/site-packages/jupyterhub/app.py", line 1865, in start
        await self.proxy.start()
      File "/usr/lib/python3.5/site-packages/jupyterhub/proxy.py", line 601, in start
        await server.wait_up(1)
      File "/usr/lib/python3.5/site-packages/jupyterhub/utils.py", line 165, in wait_for_server
        timeout=timeout
      File "/usr/lib/python3.5/site-packages/jupyterhub/utils.py", line 155, in exponential_backoff
        raise TimeoutError(fail_message)
    TimeoutError: Server at 172.16.0.20:80 didn't respond in 1 seconds
minrk commented 5 years ago

Can you try:

configurable-http-proxy --ip 172.16.0.20 --port 80 --log-level=debug

?

Can you also verify that 127.16.0.20 is a bindable ip address on the machine?

joergFLI commented 5 years ago

Hi, when run as user configurable-http-proxy --ip 172.16.0.20 --port 80 --log-level=debug gives

07:46:45.235 [ConfigProxy] warn: REST API is not authenticated. 07:46:45.251 [ConfigProxy] info: Proxying http://172.16.0.20:80 to (no default) 07:46:45.252 [ConfigProxy] info: Proxy API at http://localhost:81/api/routes 07:46:45.259 [ConfigProxy] error: Uncaught Exception 07:46:45.261 [ConfigProxy] error: Uncaught Exception

However, when run with sudo, there is no error (but the warning) When I run jupyter with sudo (not sure if I want that), it gives

sudo jupyterhub --config /home/Services/Jupyterhub/jupyterhub_config.py it gives

[I 2018-12-18 07:59:01.055 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4 [I 2018-12-18 07:59:01.056 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4 [I 2018-12-18 07:59:01.059 JupyterHub app:1016] Loading cookie_secret from /home/joerg.linde/jupyterhub_cookie_secret [I 2018-12-18 07:59:01.104 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN [W 2018-12-18 07:59:01.106 JupyterHub app:1173] No admin users, admin interface will be unavailable. [W 2018-12-18 07:59:01.107 JupyterHub app:1174] Add any administrative users to c.Authenticator.admin_users in config. [I 2018-12-18 07:59:01.107 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed. [I 2018-12-18 07:59:01.201 JupyterHub app:1855] Hub API listening on http://127.0.0.1:9000/hub/ [W 2018-12-18 07:59:01.205 JupyterHub proxy:565] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else... [I 2018-12-18 07:59:01.205 JupyterHub proxy:567] Starting proxy @ http://172.16.0.20:80/ 07:59:01.587 [ConfigProxy] info: Proxying http://172.16.0.20:80 to (no default) 07:59:01.594 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes 07:59:01.603 [ConfigProxy] error: Uncaught Exception 07:59:01.659 [ConfigProxy] info: 200 GET /api/routes [I 2018-12-18 07:59:01.659 JupyterHub proxy:301] Checking routes [I 2018-12-18 07:59:01.659 JupyterHub proxy:370] Adding default route for Hub: / => http://127.0.0.1:9000 07:59:01.666 [ConfigProxy] info: Adding route / -> http://127.0.0.1:9000 07:59:01.668 [ConfigProxy] info: 201 POST /api/routes/ [I 2018-12-18 07:59:01.669 JupyterHub app:1912] JupyterHub is now running at http://172.16.0.20:80/ 07:59:35.227 [ConfigProxy] error: 404 GET / 07:59:37.350 [ConfigProxy] error: 404 GET /

Config file lokks like that

grep "^[^#;]" /home/Services/Jupyterhub/jupyterhub_config.py

c.JupyterHub.config_file = 'jupyterhub_config.py' c.JupyterHub.cookie_max_age_days = 14 c.JupyterHub.cookie_secret_file = 'jupyterhub_cookie_secret' c.JupyterHub.data_files_path = '/usr/share/jupyter/hub' c.JupyterHub.db_url = 'sqlite:///jupyterhub.sqlite' c.JupyterHub.hub_ip = '127.0.0.1' c.JupyterHub.hub_port = 9000 c.JupyterHub.ip = '172.16.0.20' c.JupyterHub.port = 80 c.Spawner.default_url = '/lab' c.Spawner.environment = { 'JUPYTER_ENABLE_LAB': 'yes' } c.Spawner.cmd = ['jupyter-labhub']

How do I check, if 127.16.0.20 is bindable?

minrk commented 5 years ago

How do I check, if 127.16.0.20 is bindable?

try:

python3 -m http.server --bind 127.16.0.20
joergFLI commented 5 years ago

Looks good

python3 -m http.server --bind 127.16.0.20

Serving HTTP on 127.16.0.20 port 8000 ...

minrk commented 5 years ago

And just to make sure it matches, port 80 as well:

python3 -m http.server --bind 127.16.0.20 80
joergFLI commented 5 years ago

Works fine with sudo but not as user python3 -m http.server --bind 127.16.0.20 80

Traceback (most recent call last): File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main "main", mod_spec) File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/lib64/python3.4/http/server.py", line 1248, in test(HandlerClass=handler_class, port=args.port, bind=args.bind) File "/usr/lib64/python3.4/http/server.py", line 1221, in test httpd = ServerClass(server_address, HandlerClass) File "/usr/lib64/python3.4/socketserver.py", line 430, in init self.server_bind() File "/usr/lib64/python3.4/http/server.py", line 136, in server_bind socketserver.TCPServer.server_bind(self) File "/usr/lib64/python3.4/socketserver.py", line 444, in server_bind self.socket.bind(self.server_address) PermissionError: [Errno 13] Permission denied

sblack4 commented 5 years ago

I had the same issue and the fix in this issue fixed it. I just ran the below command as root 

setcap 'cap_net_bind_service=+ep' `which node`

This was my command and output. 

$ jupyterhub --debug
[D 2019-02-12 20:00:02.282 JupyterHub application:555] Looking for jupyterhub_config in /etc/jupyterhub
[D 2019-02-12 20:00:02.283 JupyterHub application:577] Loaded config file: /etc/jupyterhub/jupyterhub_config.py
[I 2019-02-12 20:00:02.284 JupyterHub app:1673] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.4
[I 2019-02-12 20:00:02.284 JupyterHub app:1673] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.4
[D 2019-02-12 20:00:02.285 JupyterHub app:1625] Could not load pycurl: No module named 'pycurl'
    pycurl is recommended if you have a large number of users.
[I 2019-02-12 20:00:02.286 JupyterHub app:1016] Loading cookie_secret from /etc/jupyterhub/jupyterhub_cookie_secret
[D 2019-02-12 20:00:02.286 JupyterHub app:1071] Connecting to db: sqlite:///jupyterhub.sqlite
[D 2019-02-12 20:00:02.300 JupyterHub orm:685] database schema version found: 896818069c98
[I 2019-02-12 20:00:02.304 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[I 2019-02-12 20:00:02.326 JupyterHub app:1201] Not using whitelist. Any authenticated user will be allowed.
[D 2019-02-12 20:00:02.334 JupyterHub app:1473] Loading state for admin from db
[D 2019-02-12 20:00:02.336 JupyterHub app:1489] Loaded users:
    admin admin
[I 2019-02-12 20:00:02.344 JupyterHub app:1855] Hub API listening on http://127.0.0.1:8081/hub/
[W 2019-02-12 20:00:02.344 JupyterHub proxy:472] Found proxy pid file: /etc/jupyterhub/jupyterhub-proxy.pid
[W 2019-02-12 20:00:02.344 JupyterHub proxy:489] Proxy still running at pid=73
[W 2019-02-12 20:00:03.350 JupyterHub proxy:509] Stopped proxy at pid=73
[D 2019-02-12 20:00:03.351 JupyterHub proxy:525] Removing proxy pid file jupyterhub-proxy.pid
[W 2019-02-12 20:00:03.353 JupyterHub proxy:565] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2019-02-12 20:00:03.353 JupyterHub proxy:567] Starting proxy @ http://:80/
[D 2019-02-12 20:00:03.353 JupyterHub proxy:568] Proxy cmd: ['configurable-http-proxy', '--ip', '', '--port', '80', '--api-ip', '127.0.0.1', '--api-port', '8001', '--error-target', 'http://127.0.0.1:8081/hub/error']
[D 2019-02-12 20:00:03.358 JupyterHub proxy:517] Writing proxy pid file: jupyterhub-proxy.pid
20:00:03.554 [ConfigProxy] info: Proxying http://*:80 to (no default)
20:00:03.558 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
20:00:03.560 [ConfigProxy] error: Uncaught Exception
[C 2019-02-12 20:00:14.561 JupyterHub app:1867] Failed to start proxy
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/jupyterhub/app.py", line 1865, in start
        await self.proxy.start()
      File "/usr/local/lib/python3.7/site-packages/jupyterhub/proxy.py", line 601, in start
        await server.wait_up(1)
      File "/usr/local/lib/python3.7/site-packages/jupyterhub/utils.py", line 165, in wait_for_server
        timeout=timeout
      File "/usr/local/lib/python3.7/site-packages/jupyterhub/utils.py", line 155, in exponential_backoff
        raise TimeoutError(fail_message)
    TimeoutError: Server at b42a1d96c6dd:80 didn't respond in 1 seconds