Log out no longer works

[jkaufman-LogRhythm]

`

Shutting down 0 terminals Apr 28 19:39:00 unixbox jupyterhub[7132]: [I 2021-04-28 19:39:00.294 JupyterHub base:1110] User testUser server took 0.568 seconds to stop Apr 28 19:39:00 unixbox jupyterhub[7132]: [I 2021-04-28 19:39:00.294 JupyterHub samlauthenticator:828] User logged out: testUser Apr 28 19:39:00 unixbox jupyterhub[7132]: [E 2021-04-28 19:39:00.299 JupyterHub web:1789] Uncaught exception GET /hub/logout (::ffff:192.168.100.10) Apr 28 19:39:00 unixbox jupyterhub[7132]: HTTPServerRequest(protocol='http', host='192.168.100.98:8000', method='GET', uri='/hub/logout', version='HTTP/1.1', remote_ip='::ffff:192.168.100.10') Apr 28 19:39:00 unixbox jupyterhub[7132]: Traceback (most recent call last): Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute Apr 28 19:39:00 unixbox jupyterhub[7132]: result = await result Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/samlauthenticator/samlauthenticator.py", line 852, in get Apr 28 19:39:00 unixbox jupyterhub[7132]: authenticator_self._get_redirect_from_metadata_and_redirect('md:SingleLogoutService', Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/samlauthenticator/samlauthenticator.py", line 730, in _get_redirect_from_metadata_and_redirect Apr 28 19:39:00 unixbox jupyterhub[7132]: handler_self.redirect(redirect_link_getter(saml_metadata_etree)[0], permanent=False) Apr 28 19:39:00 unixbox jupyterhub[7132]: IndexError: list index out of range

[distortedsignal]

Well, that's an interesting faliure.

Slightly more formatted look for my future edification:

Shutting down 0 terminals

Apr 28 19:39:00 unixbox jupyterhub[7132]:
[I 2021-04-28 19:39:00.294 JupyterHub base:1110] User testUser server took 0.568 seconds to stop 

Apr 28 19:39:00 unixbox jupyterhub[7132]: 
[I 2021-04-28 19:39:00.294 JupyterHub samlauthenticator:828] User logged out: testUser 

Apr 28 19:39:00 unixbox jupyterhub[7132]: 
[E 2021-04-28 19:39:00.299 JupyterHub web:1789] Uncaught exception GET /hub/logout (::ffff:192.168.100.10) 

Apr 28 19:39:00 unixbox jupyterhub[7132]: HTTPServerRequest(protocol='http', host='192.168.100.98:8000', method='GET', uri='/hub/logout', version='HTTP/1.1', remote_ip='::ffff:192.168.100.10') 
Apr 28 19:39:00 unixbox jupyterhub[7132]: Traceback (most recent call last): 
Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute 
Apr 28 19:39:00 unixbox jupyterhub[7132]: result = await result 
Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/samlauthenticator/samlauthenticator.py", line 852, in get 
Apr 28 19:39:00 unixbox jupyterhub[7132]: authenticator_self._get_redirect_from_metadata_and_redirect('md:SingleLogoutService', 
Apr 28 19:39:00 unixbox jupyterhub[7132]: File "/opt/jupyterhub/lib/python3.9/site-packages/samlauthenticator/samlauthenticator.py", line 730, in _get_redirect_from_metadata_and_redirect 
Apr 28 19:39:00 unixbox jupyterhub[7132]: handler_self.redirect(redirect_link_getter(saml_metadata_etree)[0], permanent=False) 
Apr 28 19:39:00 unixbox jupyterhub[7132]: IndexError: list index out of range

So from there, it looks like we're trying to deref here. https://github.com/HewlettPackard/jupyterhub-samlauthenticator/blob/a63e1411e328bcafd8f9113857057b842ff9f5df/samlauthenticator/samlauthenticator.py#L725-L729

I'm a little confused on why we're trying to redirect in the logout workflow. This redirect is usually used in the login workflow. If that redirect worked, it would (probably) kick you straight into your login workflow again, which would look like a different "logout no longer works" bug.

Could you share the (relavent parts of your) jupyterhub config file and your IdP's metadata xml?

Thanks.

[jkaufman-LogRhythm]

Sorry for the late response, here is my config with comments removed.

# Configuration file for jupyterhub.
  #------------------------------------------------------------------------------
  # myCompany Custom
  #------------------------------------------------------------------------------
  import git, os, shutil
  from pwd import getpwnam

  def clone_repo(user, repo_dir):
      git_url = 'git@github.com:myCompany/customCode.git'
      #pub_key = '/home/linuxuser/.ssh/id_ed25519.pub'
      pub_key = '/usr/local/share/jupyterhub/id_ed25519.pub'
      git_deploy_key = os.path.expanduser(pub_key)
      ssh_cmd = f'ssh -i {git_deploy_key}'

      rw_dir = '/opt/jupyterhub/etc/jupyterhub/'
      ssh_exe = os.path.join(rw_dir, 'ssh_exe.sh')
      with git.Repo.git.custom_environment(GIT_SSH=ssh_exe):
          git.Repo.remotes.origin.fetch()
      print('fetch end')
      git.Repo.clone_from(git_url, repo_dir, env={"GIT_SSH_COMMDAND": ssh_cmd})

  def set_ownership(user, repo_dir):
      uid = getpwnam(user).pw_uid
      gid = getpwnam(user).pw_gid
      print(f'uid: {uid}\ngid: {gid}\nuser: {user}')
      for root, dirs, files in os.walk(repo_dir):
          for d in dirs:
              os.chown(os.path.join(user, d), user=uid, group=gid)
          for f in files:
              os.chown(os.path.join(user, f), user=uid, group=gid)

  def create_dir_hook(spawner):
      user_name = spawner.user.name
      dir_name = os.path.join("/home", user_name)
      repo_dir = os.path.join(dir_name, 'notebooks')
      local_repo = '/home/customCode/jupyter/'
      print(repo_dir)

      if ERASE_DIR == True:
          if os.path.isdir(repo_dir):
              print('\n\nExists!\n\n')
              os.rmdir(repo_dir)
              clone_repo(user_name, repo_dir)
          else:
              print('Does not exist!')
              #shutil.rmtree(repo_dir)
              os.mkdir(repo_dir, 755)
              set_ownership(user_name, repo_dir)
              clone_repo(user_name, repo_dir)
              shutil.copytree(local_repo, repo_dir)

      if ERASE_DIR == False and not (os.path.isdir(repo_dir)):
          if os.path.isdir(repo_dir):
              os.mkdir(repo_dir)
              #clone_repo(user_name, repo_dir)
              set_ownership(user_name, repo_dir)
              shutil.copytree(local_repo, repo_dir)

      if ERASE_DIR == False and os.path.isdir(repo_dir):
          pass

  ERASE_DIR = True
  c.LocalAuthenticator.create_system_users = True
  c.Authenticator.admin_users = { 'jupyteradmin' }
  c.Spawner.pre_spawn_hook = create_dir_hook

  c.Spawner.args = ['--allow-root']

  # myCompany SAML Authentication
  # Reference : https://pypi.org/project/jupyterhub-samlauthenticator/
  # --------------------
  c.JupyterHub.authenticator_class = 'samlauthenticator.SAMLAuthenticator'
  c.SAMLAuthenticator.metadata_filepath = '/opt/jupyterhub/etc/jupyterhub/metadata.xml'

  # The IdP is sending the SAML Response in a field named 'R'
  #c.SAMLAuthenticator.login_post_field = 'R'

  # We want to make sure that we're the only one receiving this SAML Response
  #c.SAMLAuthenticator.audience = 'http://192.168.100.98:8000'
  #c.SAMLAuthenticator.recipient = 'http://192.168.100.98:8000/hub/login'

  # A field was placed in the SAML Response that contains the user's first name and last name separated by a period.
  # Let's use that for the username.
  #c.SAMLAuthenticator.xpath_username_location = '//saml:Attribute[@Name="DottedName"]/saml:AttributeValue/text()'

  # Path to the group/role membership in the SAML response.
  c.SAMLAuthenticator.xpath_role_location = '//saml:Attribute[@Name="Roles"]/saml:AttributeValue/text()'

  # REQUIRED
  # needs: '2021-04-26T16:37:24.853Z' does not match format '%Y-%m-%dT%H:%M:%SZ'
  # https://stackoverflow.com/questions/14527896/string-to-time-with-decimal-seconds
  c.SAMLAuthenticator.time_format_string = '%Y-%m-%dT%H:%M:%S.%fZ'
  #c.SAMLAuthenticator.time_format_string = '%a %B %d, %Y %H:%M%S'
  c.SAMLAuthenticator.nameid_format = 'urn:oasis:names:tc:SAML:2.0:nameid-format:persistent'

  # Looks like we can't get the timezone from the previous string - we need to set it
  #c.SAMLAuthenticator.idp_timezone = 'US/Mountain'

  # Shutdown all servers when the user logs out
  c.SAMLAuthenticator.shutdown_on_logout = True

  # Don't send the user to the SLO address on logout
  c.SAMLAuthenticator.slo_forward_on_logout = True

  # Because the entity id isn't a url, we need to set the acs endpoint url
  #c.SAMLAuthenticator.acs_endpoint_url = 'http://192.168.100.98:8000/hub/login'

  ## The URL the single-user server should start in.
  #
  #
  #  Example uses:
  #
  #  - You can set `notebook_dir` to `/` and `default_url` to `/tree/home/{username}` to allow people to
  #    navigate the whole filesystem from their notebook server, but still start in their home directory.
  #  - Start with `/notebooks` instead of `/tree` if `default_url` points to a notebook instead of a directory.
  #  - You can set this to `/lab` to have JupyterLab start by default, rather than Jupyter Notebook.
  #  Default: ''
  # c.Spawner.default_url = ''
  c.Spawner.default_url = '/lab'
  #c.Spawner.default_url = '/home/{username}'
  def notebook_dir_hook(spawner):
      # change to whatever you'd like, but this is the default value I think, and the spawner
      # will be able to assume `"{username}"` means the name of the current user
      # NOTE: if you always want to have the same string, then simply set this value once in
      # the extra config for JupyterHub by adding (to hub.extraConfig):
      # c.KubeSpawner.notebook_dir = "/user/home/{username}"
      spawner.notebook_dir = "/home/{username}"

  c.Spawner.pre_spawn_hook = notebook_dir_hook

[distortedsignal]

Ok, I see that you're using the forward_on_logout field - would you mind sending me the file at /opt/jupyterhub/etc/jupyterhub/metadata.xml so I can see where your users should be forwarded?

[jkaufman-LogRhythm]

Sorry for the late response, been swamped, looking at this I feel I may need to alter the redirect:

<md:EntityDescriptor xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" entityID="http://www.okta.com/someRandomID">
<md:IDPSSODescriptor WantAuthnRequestsSigned="false" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
<md:KeyDescriptor use="signing">
<ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:X509Data>
<ds:X509Certificate>Redacted</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
<md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat>
<md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://dev-number.okta.com/app/dev-number_jupyterlab_1/someRandomID/sso/saml"/>
<md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://dev-number.okta.com/app/dev-number_jupyterlab_1/someRandomID/sso/saml"/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>

[distortedsignal]

Yeah, your metadata doesn't have a URL for your users to be redirected to on Logout.

That's usually going to look something like <md:SingleLogoutService>. Since your XML metadata doesn't have one of those, your users aren't getting forwarded on logout. You can turn off forward on logout and your users should go to the JupyterHub logout screen with the option to log in again. That "re-login" workflow should work just like the first login.