search

jupyterlab / jupyterlab-git

A Git extension for JupyterLab
BSD 3-Clause "New" or "Revised" License
1.45k stars 313 forks source link

After first commit, jupyterLab, Hub... under Ubuntu 20.04 #845

Closed spouyllau closed 3 years ago

spouyllau commented 3 years ago

Hello, On Ubuntu 20.04, JupyterHub and Lab, after the first commit, all become slow and re-login is impossible. I have in log : 

[E 2020-12-17 15:49:05.190 SingleUserNotebookApp log:181] 500 POST /user/spouyllau/git/branch?1608216525092 (spouyllau@::ffff:195.98.225.117) 20003.99ms
[E 2020-12-17 15:49:18.595 JupyterHub web:1789] Uncaught exception GET /hub/oauth_callback?code=f29b9cd1df7fc35222e049f2c525af3a46e29cf3af57c335ba3da6c9bb720540&state=eyJzdGF0ZV9pZCI6ICIzNzgzMzAzZjg1YWQ0MzQ5ODQ0NTQwYWNjN2RlMWQzOCIsICJuZXh0X3VybCI6ICIifQ%3D%3D&session_state=6Q%2F7OqcSFV%2Fy3zgofWTtFnyW1wZaeZKomlRDX1YRChM%3D.Z2h1ZHVRZ0UxUGpwcWFDZXE2ai9PblpCTDREQko5WjVVZkRENjNqVDNjWVlDK0V3UHN4N3YyQy9MTDNFZjAyelRJYlQ1alpHTWRHQm0wczZ2bHl5dzdrN3NXMmRjSWZYWkxRVW11cnZOalZSSzJlaVBtOGtlLytFK2NYb1BJT3o (::ffff:195.98.225.117)
    HTTPServerRequest(protocol='https', host='callisto.huma-num.fr', method='GET', uri='/hub/oauth_callback?code=f29b9cd1df7fc35222e049f2c525af3a46e29cf3af57c335ba3da6c9bb720540&state=eyJzdGF0ZV9pZCI6ICIzNzgzMzAzZjg1YWQ0MzQ5ODQ0NTQwYWNjN2RlMWQzOCIsICJuZXh0X3VybCI6ICIifQ%3D%3D&session_state=6Q%2F7OqcSFV%2Fy3zgofWTtFnyW1wZaeZKomlRDX1YRChM%3D.Z2h1ZHVRZ0UxUGpwcWFDZXE2ai9PblpCTDREQko5WjVVZkRENjNqVDNjWVlDK0V3UHN4N3YyQy9MTDNFZjAyelRJYlQ1alpHTWRHQm0wczZ2bHl5dzdrN3NXMmRjSWZYWkxRVW11cnZOalZSSzJlaVBtOGtlLytFK2NYb1BJT3o', version='HTTP/1.1', remote_ip='::ffff:195.98.225.117')
    Traceback (most recent call last):
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute
        result = await result
      File "/opt/jupyterhub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 224, in get
        user = await self.login_user()
      File "/opt/jupyterhub/lib/python3.8/site-packages/jupyterhub/handlers/base.py", line 749, in login_user
        authenticated = await self.authenticate(data)
      File "/opt/jupyterhub/lib/python3.8/site-packages/jupyterhub/auth.py", line 462, in get_authenticated_user
        authenticated = await maybe_future(self.authenticate(handler, data))
      File "/opt/jupyterhub/lib/python3.8/site-packages/oauthenticator/generic.py", line 155, in authenticate
        token_resp_json = await self._get_token(http_client, headers, params)
      File "/opt/jupyterhub/lib/python3.8/site-packages/oauthenticator/generic.py", line 97, in _get_token
        resp = await http_client.fetch(req)
    tornado.simple_httpclient.HTTPTimeoutError: Timeout while connecting

[D 2020-12-17 15:49:18.598 JupyterHub base:1280] No template for 500
[E 2020-12-17 15:49:18.619 JupyterHub log:173] {
      "X-Forwarded-Host": "callisto.huma-num.fr",
      "X-Forwarded-Proto": "https",
      "X-Forwarded-Port": "443",
      "X-Forwarded-For": "::ffff:195.98.225.117",
      "Accept-Encoding": "gzip, deflate, br",
      "Referer": "https://callisto.huma-num.fr/hub/login",
      "Accept-Language": "fr-fr",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Safari/605.1.15",
      "Cookie": "oauthenticator-state=[secret]; lemonldap=[secret]; _ga=[secret]; _xsrf=[secret]; rl_trait=[secret]; rl_user_id=[secret]; rl_anonymous_id=[secret]; experimentation_subject_id=[secret]",
      "Connection": "close",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
      "Host": "callisto.huma-num.fr"
    }
[E 2020-12-17 15:49:18.620 JupyterHub log:181] 500 GET /hub/oauth_callback?code=[secret]&state=[secret]&session_state=[secret] (@::ffff:195.98.225.117) 20029.92ms
[E 2020-12-17 15:49:38.633 JupyterHub ioloop:761] Exception in callback functools.partial(<function _HTTPConnection.__init__.<locals>.<lambda> at 0x7f7263ca4700>, <Task finished name='Task-151' coro=<_HTTPConnection.run() done, defined at /opt/jupyterhub/lib/python3.8/site-packages/tornado/simple_httpclient.py:293> exception=gaierror(-3, 'Temporary failure in name resolution')>)
    Traceback (most recent call last):
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
        ret = callback()
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/simple_httpclient.py", line 290, in <lambda>
        gen.convert_yielded(self.run()), lambda f: f.result()
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/simple_httpclient.py", line 338, in run
        stream = await self.tcp_client.connect(
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/tcpclient.py", line 265, in connect
        addrinfo = await self.resolver.resolve(host, port, af)
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/netutil.py", line 398, in resolve
        result = await IOLoop.current().run_in_executor(
      File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/opt/jupyterhub/lib/python3.8/site-packages/tornado/netutil.py", line 382, in _resolve_addr
        addrinfo = socket.getaddrinfo(host, port, family, socket.SOCK_STREAM)
      File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
        for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
    socket.gaierror: [Errno -3] Temporary failure in name resolution

Have you idea ?

fcollonval commented 3 years ago

Thanks for reporting @spouyllau

Do you have rights to change JupyterLab server settings?

if yes, could you set the debug mode on by setting the Application.log_level to 'DEBUG' https://jupyter-notebook.readthedocs.io/en/stable/config.html

Then should get in the log the git command output.

But from the error you are showing above, I fear you are even not reaching the git server extension and something is weird with the Hub itself - it fails to validate your credentials (what does not make sense except if they have expired). So this looks more like a problem with JupyterHub.