403 login error when URL has a folder (cookies not found)

wmay commented 1 year ago

Problem description

I get a 403 error trying to log in to a remote jupyterlab with a folder in the URL, like example.com/folder. Jupyterlab sends a response complaining that '_xsrf' argument missing from POST.

I finally got it to work by altering ein:query-prepare-header:

(defun ein:query-prepare-header (url settings &optional securep)
  "Ensure that REST calls to the jupyter server have the correct _xsrf argument."
  (let* ((host (url-host (url-generic-parse-url url)))
         ;;(cookies (request-cookie-alist host "/" securep))
         (cookies (request-cookie-alist host "/folder/" securep)) ;; <-- changed
         (xsrf (or (cdr (assoc-string "_xsrf" cookies))
                   (gethash host ein:query-xsrf-cache)))
         (key (ein:query-divine-authorization-tokens-key url))
         (token (aand key
                      (gethash key ein:query-authorization-tokens)
                      (cons "Authorization" (format "token %s" it)))))
    (setq settings (plist-put settings :headers
                              (append (plist-get settings :headers)
                                      (list (cons "User-Agent" "Mozilla/5.0")))))
    (when token
      (setq settings (plist-put settings :headers
                                (append (plist-get settings :headers)
                                        (list token)))))
    (when xsrf
      (setq settings (plist-put settings :headers
                                (append (plist-get settings :headers)
                                        (list (cons "X-XSRFTOKEN" xsrf)))))
      (setf (gethash host ein:query-xsrf-cache) xsrf))
    (setq settings (plist-put settings :encoding 'binary))
    settings))

Steps to reproduce the problem

Try to log in to a jupyterlab server with a folder in the URL.

System info:

("EIN system info"
 :emacs-version
 "GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
 of 2022-01-24, modified by Debian"
 :window-system x
 :emacs-variant nil
 :build
 "--build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd --with-pop=yes --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp --with-sound=alsa --without-gconf --with-mailutils --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd --with-pop=yes --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp --with-sound=alsa --without-gconf --with-mailutils --with-cairo --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2 -ffile-prefix-map=/build/emacs-NbbgEv/emacs-27.1+1=. -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'"
 :os
 (:uname
  "Linux will-Inspiron-7520 5.19.0-45-generic #46~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 7 15:06:04 UTC 20 x86_64 x86_64 x86_64 GNU/Linux
"
  :lsb-release
  "No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:    22.04
Codename:   jammy
")
 :jupyter
 "Selected Jupyter core packages...
IPython          : 7.31.1
ipykernel        : 6.7.0
ipywidgets       : 6.0.0
jupyter_client   : 7.1.2
jupyter_core     : 4.9.1
jupyter_server   : not installed
jupyterlab       : not installed
nbclient         : 0.5.6
nbconvert        : 6.4.0
nbformat         : 5.1.3
notebook         : 6.4.8
qtconsole        : not installed
traitlets        : 5.1.1
"
 :image-types
 (svg png gif tiff jpeg xpm xbm pbm)
 :image-types-available
 (svg png gif tiff jpeg xpm xbm pbm)
 :request-backend curl
 :ein
 (:version "20230426.1857"
       :source-dir "/home/will/.emacs.d/elpa/ein-20230426.1857/")
 :lib
 ((:name "websocket"
     :path "~/.emacs.d/elpa/websocket-20230305.410/websocket.elc"
     :featurep t
     :version-var websocket-version
     :version "1.12")
  (:name "anaphora"
     :path "~/.emacs.d/elpa/anaphora-20180618.2200/anaphora.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "request"
     :path "~/.emacs.d/elpa/request-20230127.417/request.elc"
     :featurep t
     :version-var request-version
     :version "0.3.3")
  (:name "deferred"
     :path "~/.emacs.d/elpa/deferred-20170901.1330/deferred.elc"
     :featurep t
     :version-var deferred:version
     :version "0.5.0")
  (:name "polymode"
     :path "~/.emacs.d/elpa/polymode-20230317.1218/polymode.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "dash"
     :path "~/.emacs.d/elpa/dash-20230617.2046/dash.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "with-editor"
     :path "~/.emacs.d/elpa/with-editor-20230608.1237/with-editor.elc"
     :featurep nil
     :version-var nil
     :version nil)))

Logs:

request-log:

[debug] request--curl: --silent --location --cookie /home/will/.emacs.d/request/curl-cookie-jar --cookie-jar /home/will/.emacs.d/request/curl-cookie-jar --include --write-out \n(:num-redirects %{num_redirects} :url-effective "%{url_effective}") --junk-session-cookies --trace-ascii /tmp/curl-trace --compressed --header User-Agent: Mozilla/5.0 --url https://example.com/folder/login
[debug] request--curl-callback: event finished

[debug] request--callback: UNPARSED
HTTP/2 200 
date: Thu, 22 Jun 2023 00:59:28 GMT
content-type: text/html; charset=UTF-8
content-length: 3724
x-content-type-options: nosniff
content-security-policy: frame-ancestors 'self'; report-uri /folder/api/security/csp-report
access-control-allow-origin: *
etag: "8b341139730b26e06258966df39b122bd9374af7"
strict-transport-security: max-age=15724800; includeSubDomains

<!DOCTYPE HTML>
<html>

<head>

    <meta charset="utf-8">

    <title>Jupyter Server</title>
    <link id="favicon" rel="shortcut icon" type="image/x-icon" href="/folder/static/favicon.ico?v=50afa725b5de8b00030139d09b38620224d4e7dba47c07ef0e86d4643f30c9bfe6bb7e1a4a1c561aa32834480909a4b6fe7cd1e17f7159330b6b5914bf45a880">

    <link rel="stylesheet" href="/folder/static/style/bootstrap.min.css?v=0e8a7fbd6de23ad6b27ab95802a0a0915af6693af612bc304d83af445529ce5d95842309ca3405d10f538d45c8a3a261b8cff78b4bd512dd9effb4109a71d0ab" />
    <link rel="stylesheet" href="/folder/static/style/bootstrap-theme.min.css?v=8b2f045cb5b4d5ad346f6e816aa2566829a4f5f2783ec31d80d46a57de8ac0c3d21fe6e53bcd8e1f38ac17fcd06d12088bc9b43e23b5d1da52d10c6b717b22b3" />
    <link rel="stylesheet" href="/folder/static/style/index.css?v=30372e3246a801d662cf9e3f9dd656fa192eebde9054a2282449fe43919de9f0ee9b745d7eb49d3b0a5e56357912cc7d776390eddcab9dac85b77bdb17b4bdae" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">

</head>

<body class=""    dir="ltr">

  <noscript>
    <div id='noscript'>
      Jupyter Server requires JavaScript.<br>
      Please enable it to proceed. 
    </div>
  </noscript>

  <div id="header" role="navigation" aria-label="Top Menu">
    <div id="header-container" class="container">
      <div id="jupyter_server" class="nav navbar-brand"><a href="/folder/lab" title='dashboard'>
          <img src='/folder/static/logo/logo.png?v=a2a176ee3cee251ffddf5fa21fe8e43727a9e5f87a06f9c91ad7b776d9e9d3d5e0159c16cc188a3965e00375fb4bc336c16067c688f5040c0c2d4bfdb852a9e4' alt='Jupyter Server' />
        </a></div>

    </div>
    <div class="header-bar"></div>

  </div>

  <div id="site">

<div id="jupyter-main-app" class="container">

    <div class="row">
        <div class="navbar col-sm-8">
            <div class="navbar-inner">
                <div class="container">
                    <div class="center-nav">
                        <form action="/folder/login?next=%2Ffolder%2F" method="post" class="navbar-form pull-left">
                            <input type="hidden" name="_xsrf" value="2|1d6fddb0|69fc931490bf0aca4a5f6bb040f6e350|1686971138"/>

                            <label for="password_input"><strong>Password:</strong></label>

                            <input type="password" name="password" id="password_input" class="form-control">
                            <button type="submit" class="btn btn-default" id="login_submit">Log in</button>
                        </form>
                    </div>
                </div>
            </div>
        </div>
    </div>

</div>

  </div>

  <script type='text/javascript'>
    function _remove_token_from_url() {
      if (window.location.search.length <= 1) {
        return;
      }
      var search_parameters = window.location.search.slice(1).split('&');
      for (var i = 0; i < search_parameters.length; i++) {
        if (search_parameters[i].split('=')[0] === 'token') {
          // remote token from search parameters
          search_parameters.splice(i, 1);
          var new_search = '';
          if (search_parameters.length) {
            new_search = '?' + search_parameters.join('&');
          }
          var new_url = window.location.origin +
            window.location.pathname +
            new_search +
            window.location.hash;
          window.history.replaceState({}, "", new_url);
          return;
        }
      }
    }
    _remove_token_from_url();
  </script>
</body>

</html>
[debug] request--callback: PARSED
(:reprompt t)
[debug] request--callback: executing success
[debug] request--curl: --silent --location --cookie /home/will/.emacs.d/request/curl-cookie-jar --cookie-jar /home/will/.emacs.d/request/curl-cookie-jar --include --write-out \n(:num-redirects %{num_redirects} :url-effective "%{url_effective}") --trace-ascii /tmp/curl-trace --compressed --header User-Agent: Mozilla/5.0 --url https://example.com/folder/login --data-binary @-
[debug] request--callback: executing complete
[debug] request--curl-callback: event finished

[debug] request--callback: UNPARSED
HTTP/2 403 
date: Thu, 22 Jun 2023 00:59:36 GMT
content-type: text/html
content-length: 3100
x-content-type-options: nosniff
content-security-policy: frame-ancestors 'self'; report-uri /folder/api/security/csp-report
access-control-allow-origin: *
strict-transport-security: max-age=15724800; includeSubDomains

<!DOCTYPE HTML>
<html>

<head>

    <meta charset="utf-8">

    <title>Jupyter Server</title>
    <link id="favicon" rel="shortcut icon" type="image/x-icon" href="/folder/static/favicon.ico?v=50afa725b5de8b00030139d09b38620224d4e7dba47c07ef0e86d4643f30c9bfe6bb7e1a4a1c561aa32834480909a4b6fe7cd1e17f7159330b6b5914bf45a880">

    <link rel="stylesheet" href="/folder/static/style/bootstrap.min.css?v=0e8a7fbd6de23ad6b27ab95802a0a0915af6693af612bc304d83af445529ce5d95842309ca3405d10f538d45c8a3a261b8cff78b4bd512dd9effb4109a71d0ab" />
    <link rel="stylesheet" href="/folder/static/style/bootstrap-theme.min.css?v=8b2f045cb5b4d5ad346f6e816aa2566829a4f5f2783ec31d80d46a57de8ac0c3d21fe6e53bcd8e1f38ac17fcd06d12088bc9b43e23b5d1da52d10c6b717b22b3" />
    <link rel="stylesheet" href="/folder/static/style/index.css?v=30372e3246a801d662cf9e3f9dd656fa192eebde9054a2282449fe43919de9f0ee9b745d7eb49d3b0a5e56357912cc7d776390eddcab9dac85b77bdb17b4bdae" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">

<style type="text/css">
    /* disable initial hide */
    div#header,
    div#site {
        display: block;
    }
</style>

</head>

<body class=""    dir="ltr">

  <noscript>
    <div id='noscript'>
      Jupyter Server requires JavaScript.<br>
      Please enable it to proceed. 
    </div>
  </noscript>

  <div id="header" role="navigation" aria-label="Top Menu">
    <div id="header-container" class="container">
      <div id="jupyter_server" class="nav navbar-brand"><a href="/folder/lab" title='dashboard'>
          <img src='/folder/static/logo/logo.png?v=a2a176ee3cee251ffddf5fa21fe8e43727a9e5f87a06f9c91ad7b776d9e9d3d5e0159c16cc188a3965e00375fb4bc336c16067c688f5040c0c2d4bfdb852a9e4' alt='Jupyter Server' />
        </a></div>

    </div>
    <div class="header-bar"></div>

  </div>

  <div id="site">

<div class="error">

    <h1>403 : Forbidden</h1>

    <p>The error was:</p>
    <div class="traceback-wrapper">
        <pre class="traceback">&#39;_xsrf&#39; argument missing from POST</pre>
    </div>

</div>

  </div>

  <script type='text/javascript'>
    function _remove_token_from_url() {
      if (window.location.search.length <= 1) {
        return;
      }
      var search_parameters = window.location.search.slice(1).split('&');
      for (var i = 0; i < search_parameters.length; i++) {
        if (search_parameters[i].split('=')[0] === 'token') {
          // remote token from search parameters
          search_parameters.splice(i, 1);
          var new_search = '';
          if (search_parameters.length) {
            new_search = '?' + search_parameters.join('&');
          }
          var new_url = window.location.origin +
            window.location.pathname +
            new_search +
            window.location.hash;
          window.history.replaceState({}, "", new_url);
          return;
        }
      }
    }
    _remove_token_from_url();
  </script>
</body>

</html>
[error] request--callback: peculiar error: 403
[debug] request--callback: executing error
[debug] request--curl: --silent --location --cookie /home/will/.emacs.d/request/curl-cookie-jar --cookie-jar /home/will/.emacs.d/request/curl-cookie-jar --include --write-out \n(:num-redirects %{num_redirects} :url-effective "%{url_effective}") --trace-ascii /tmp/curl-trace --compressed --header User-Agent: Mozilla/5.0 --url https://example.com/folder/login --data-binary @-
[debug] request--callback: executing complete
[debug] request--curl-callback: event finished

[debug] request--callback: UNPARSED
HTTP/2 403 
date: Thu, 22 Jun 2023 00:59:36 GMT
content-type: text/html
content-length: 3100
x-content-type-options: nosniff
content-security-policy: frame-ancestors 'self'; report-uri /folder/api/security/csp-report
access-control-allow-origin: *
strict-transport-security: max-age=15724800; includeSubDomains

<!DOCTYPE HTML>
<html>

<head>

    <meta charset="utf-8">

    <title>Jupyter Server</title>
    <link id="favicon" rel="shortcut icon" type="image/x-icon" href="/folder/static/favicon.ico?v=50afa725b5de8b00030139d09b38620224d4e7dba47c07ef0e86d4643f30c9bfe6bb7e1a4a1c561aa32834480909a4b6fe7cd1e17f7159330b6b5914bf45a880">

    <link rel="stylesheet" href="/folder/static/style/bootstrap.min.css?v=0e8a7fbd6de23ad6b27ab95802a0a0915af6693af612bc304d83af445529ce5d95842309ca3405d10f538d45c8a3a261b8cff78b4bd512dd9effb4109a71d0ab" />
    <link rel="stylesheet" href="/folder/static/style/bootstrap-theme.min.css?v=8b2f045cb5b4d5ad346f6e816aa2566829a4f5f2783ec31d80d46a57de8ac0c3d21fe6e53bcd8e1f38ac17fcd06d12088bc9b43e23b5d1da52d10c6b717b22b3" />
    <link rel="stylesheet" href="/folder/static/style/index.css?v=30372e3246a801d662cf9e3f9dd656fa192eebde9054a2282449fe43919de9f0ee9b745d7eb49d3b0a5e56357912cc7d776390eddcab9dac85b77bdb17b4bdae" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">

<style type="text/css">
    /* disable initial hide */
    div#header,
    div#site {
        display: block;
    }
</style>

</head>

<body class=""    dir="ltr">

  <noscript>
    <div id='noscript'>
      Jupyter Server requires JavaScript.<br>
      Please enable it to proceed. 
    </div>
  </noscript>

  <div id="header" role="navigation" aria-label="Top Menu">
    <div id="header-container" class="container">
      <div id="jupyter_server" class="nav navbar-brand"><a href="/folder/lab" title='dashboard'>
          <img src='/folder/static/logo/logo.png?v=a2a176ee3cee251ffddf5fa21fe8e43727a9e5f87a06f9c91ad7b776d9e9d3d5e0159c16cc188a3965e00375fb4bc336c16067c688f5040c0c2d4bfdb852a9e4' alt='Jupyter Server' />
        </a></div>

    </div>
    <div class="header-bar"></div>

  </div>

  <div id="site">

<div class="error">

    <h1>403 : Forbidden</h1>

    <p>The error was:</p>
    <div class="traceback-wrapper">
        <pre class="traceback">&#39;_xsrf&#39; argument missing from POST</pre>
    </div>

</div>

  </div>

  <script type='text/javascript'>
    function _remove_token_from_url() {
      if (window.location.search.length <= 1) {
        return;
      }
      var search_parameters = window.location.search.slice(1).split('&');
      for (var i = 0; i < search_parameters.length; i++) {
        if (search_parameters[i].split('=')[0] === 'token') {
          // remote token from search parameters
          search_parameters.splice(i, 1);
          var new_search = '';
          if (search_parameters.length) {
            new_search = '?' + search_parameters.join('&');
          }
          var new_url = window.location.origin +
            window.location.pathname +
            new_search +
            window.location.hash;
          window.history.replaceState({}, "", new_url);
          return;
        }
      }
    }
    _remove_token_from_url();
  </script>
</body>

</html>
[error] request--callback: peculiar error: 403
[debug] request--callback: executing error
[debug] request--callback: executing complete

ein:log-all:

20:59:19:705: [warn] ein:jupyter-default-kernel: (json-unknown-keyword fish) @#<buffer *scratch*>
20:59:27:874: [debug] Login attempt #-1 in response to nil from https://example.com/folder. @#<buffer *scratch*>
20:59:36:014: [debug] Login attempt #0 in response to 200 from https://example.com/folder. @#<buffer *scratch*>
20:59:36:018: [debug] ein:notebooklist-login--complete STATUS: 200 DATA: (:reprompt t) @#<buffer *scratch*>
20:59:36:309: [debug] Login attempt #1 in response to 403 from https://example.com/folder. @#<buffer *scratch*>
20:59:36:311: [debug] ein:notebooklist-login--complete STATUS: 403 DATA: nil @#<buffer *scratch*>
20:59:36:664: [error] Login to https://example.com/folder failed, error-thrown (error http 403), raw-header HTTP/2 403 
date: Thu, 22 Jun 2023 00:59:36 GMT
content-type: text/html
content-length: 3100
x-content-type-options: nosniff
content-security-policy: frame-ancestors 'self'; report-uri /folder/api/security/csp-report
access-control-allow-origin: *
strict-transport-security: max-age=15724800; includeSubDomains
 @#<buffer *scratch*>
20:59:36:685: [debug] ein:notebooklist-login--complete STATUS: 403 DATA: nil @#<buffer *scratch*>
20:59:57:138: [warn] ein:dev-packages: Don’t call me! @#<buffer  *temp*>

dickmao commented 1 year ago

That's a bug. Thanks.

Commit f47d1f6.

wmay commented 1 year ago

Oh, this still has a minor issue. The "login" part of the URL needs to be removed. Thanks for getting to this so quickly though!

I still get the 403 error. I added a bunch of messages to ein:query-prepare-header to print out the variables, after everything is assigned by let*. Here's what I got:

host is example.com
path is /folder/login
`(zerop (length path))` is nil
`(file-name-as-directory path)` is /folder/login/
cookies is nil
xsrf is nil

dickmao commented 1 year ago

I'm afraid the beatings will continue until morale improves.

Commit 8838910 is another try at this.

wmay commented 1 year ago

Still getting a 403 error. Here are the values I'm getting within ein:query-prepare-header now:

host is example.com
path is nil
`(ein:notebooklist-parse-nbpath url)` is nil
`(ein:notebooklist-keys)` is nil
`(stringp path)` is nil
cookies is nil
xsrf is nil

I also got the error (from ein:notebooklist-parse-nbpath I think):

Error (ein): https://example.com/folder/login not among: nil

dickmao commented 1 year ago

Okay, I resorted to blindly matching all of "/folder/login/", "/folder/" and "/" until it finds an xsrf cookie. Commit 32e2681.

This brute-force approach should only apply the first go-round, after which ein:notebooklist-parse-nbpath will have registered "/folder/" as the canonical url.

wmay commented 1 year ago

That fixed it, I'm able to log in now. Thanks!

millejoh / emacs-ipython-notebook