Request for users to test #127

[nnicandro]

Before I merge #127, is there anyone out there who can test it with their own notebook servers and confirm that they are able to connect a REPL to a notebook server and maybe play around with the TRAMP integration to see if there are any obvious problems.

[dcherian]

I can. My main use-case is connecting to a remote notebook server through a tunnel. I have the tunnel setup and I can access notebooks in my browser at localhost:8888 . Sometimes I have to paste in a token to login.

Some notes on jupyter-server-list-kernels

1. With no kernels started I get

   No kernels at http://localhost:8888; launch one? (y or n) y helm-M-x: Jupyter REST API error: 403, "'_xsrf' argument missing from POST"

2. After starting a kernel using the browser

   error in process filter: zmq--subprocess-filter: Error in ZMQ subprocess: jupyter-api-login-failed, ("Can't authenticate non-interactively") error in process filter: Error in ZMQ subprocess: jupyter-api-login-failed, ("Can't authenticate non-interactively") Timeout when connecting websocket to kernel id ca8d8157-6313-42b7-b6da-93a4e9ecd70a

I'm having similar login issues with local notebook servers. It connects initially. But then I can't connect to an existing kernel. It refuses to accept the token the second time.

EDIT: To be clear, After I start a kernel through the tunnel using the browser. I see a list of kernels. But hitting RET on a kernel in that list gives the jupyter-api-login-failed error. C-RET also fails with

finalizer failed: (jupyter-api-http-error 403 "'_xsrf' argument missing from POST") Jupyter REST API error: 403, "'_xsrf' argument missing from POST"

[dcherian]

Hmmm... I guess it asks me for a websocket url and that probably doesn't work for the SSH tunnel?

[dcherian]

Here's a debug log from a local kernel:

[D 10:30:12.606 NotebookApp] Accepting token-authenticated connection from 127.0.0.1 [D 10:30:12.607 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/share/jupyter/kernels [D 10:30:12.613 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 7.58ms [D 10:30:14.908 NotebookApp] Accepting token-authenticated connection from 127.0.0.1 [D 10:30:14.908 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/share/jupyter/kernels [D 10:30:14.909 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 1.85ms [D 10:30:15.683 NotebookApp] Accepting token-authenticated connection from 127.0.0.1 [D 10:30:15.684 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/share/jupyter/kernels [D 10:30:15.685 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 2.50ms [D 10:30:15.684 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/share/jupyter/kernels [D 10:30:15.685 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 2.50ms [D 10:30:45.011 NotebookApp] 200 GET /api/terminals?=1560443203218 (127.0.0.1) 0.50ms [D 10:30:45.013 NotebookApp] 200 GET /api/sessions?=1560443203217 (127.0.0.1) 1.55ms [D 10:30:45.063 NotebookApp] 200 GET /api/contents?type=directory&_=1560443203219 (127.0.0.1) 3.47ms

emacs-jupyter prompted me for a token three times and told me that I had reached max authentication tries. It seems to have succeeded though...

I'm also seeing a bunch of these

[I 10:32:22.465 NotebookApp] Malformed HTTP message from 127.0.0.1: no colon in header line

EDIT: I'm using jupyter 4.4.0 and notebook 5.7.4

[nnicandro]

@dcherian Ok great, this is what I was hoping for. What is your OS and Emacs version as well? The malformed HTTP messages may have something to do with the Emacs version. Could you evaluate (setq url-debug 1), try again, and show the HTTP headers that are sent to the server. They should be in the *Messages* buffer.

[nnicandro]

I guess it asks me for a websocket url and that probably doesn't work for the SSH tunnel?

It will as long as the tunnel can handle requests that use the WS HTTP method. I'm not sure about an SSH tunnel specifically. I did run a notebook through a proxy and had to explicitly enable WS support for it to work so maybe the SSH tunnel doesn't know how to handle WS by default.

I would think that it would since a tunnel is just redirecting bits to some other location and not concerned about the specific protocol.

[dcherian]

I reinstalled with the latest version of that branch. I start a local server with jupyter notebook.

jupyter-server-list-kernels successfully uses the token to log in (yay!). Then asks me if I want to create a server. I say y and choose Python 3. Now it says "spinning waiting for headers". The jupyter-kernels buffer shows one python3 kernel with state Starting.

Log from server:

D 09:14:17.229 NotebookApp] Accepting token-authenticated connection from 127.0.0.1
[D 09:14:17.230 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/envs/dcpy/share/jupyter/kernels
[D 09:14:17.233 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 4.34ms
[D 09:14:17.263 NotebookApp] 200 GET /api/kernels (127.0.0.1) 0.94ms
y[D 09:14:35.559 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/envs/dcpy/share/jupyter/kernels
[D 09:14:35.560 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 1.43ms
[D 09:14:37.116 NotebookApp] Starting kernel: ['/Users/dcherian/anaconda3/envs/dcpy/bin/python', '-m', 'ipykernel_launcher', '-f', '/Users/dcherian/Library/Jupyter/runtime/kernel-4eab4943-fb7d-4f30-b517-4c0cfac340a5.json']
[D 09:14:37.121 NotebookApp] Connecting to: tcp://127.0.0.1:59283
[D 09:14:37.127 NotebookApp] Connecting to: tcp://127.0.0.1:59281
[I 09:14:37.128 NotebookApp] Kernel started: 4eab4943-fb7d-4f30-b517-4c0cfac340a5
[D 09:14:37.128 NotebookApp] Kernel args: {'kernel_name': 'python3'}
[D 09:14:37.129 NotebookApp] 201 POST /api/kernels (127.0.0.1) 36.99ms
[D 09:14:37.140 NotebookApp] 200 GET /api/kernels (127.0.0.1) 0.97ms
[D 09:14:37.152 NotebookApp] 200 GET /api/kernels (127.0.0.1) 0.80ms
[D 09:14:39.899 NotebookApp] activity on 4eab4943-fb7d-4f30-b517-4c0cfac340a5: stream
[D 09:14:40.502 NotebookApp] activity on 4eab4943-fb7d-4f30-b517-4c0cfac340a5: status

Log from emacs:

Configuring package jupyter...done (0.064s)
Token authenticated [http://localhost:8888]? (y or n) y
Quit
Jupyter REPL (*jupyter-repl[python 3.7.1]-/jpy:localhost#8888/*) still connected. Kill it? (y or n) y
Shutdown the client's kernel? (y or n) y
python3 kernel shutting down...done
Configuring package crux...done
Unable to load color "#eee8d5"
Unable to load color "#859900"
Unable to load color "#cccec4"
Unable to load color "#eee8d5"
Auto-saving...done
Quit
1 (#o1, #x1, ?\C-a)
Quit
funcall-interactively: End of buffer
Contacting host: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernelspecs HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a
Authorization: token 97e86cf7449e3df21155fb09dc4ac796607c10b135c63089
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Found 1 Set-Cookie headers
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernels HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
No kernels at http://localhost:8888; launch one? (y or n) y
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernelspecs HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
POST /api/kernels HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
Content-Type: application/json
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977
Content-length: 18

{"name":"python3"}
Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=201
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernels HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernels HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/kernelspecs HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/contents/?content=1 HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Spinning waiting for headers...
Calling after change function ‘url-http-content-length-after-change-function’ for ‘#<process localhost>’
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
PUT /api/contents/tramp.kN8HIA HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
Content-Type: application/json
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977
Content-length: 44

{"content":"","type":"file","format":"text"}
Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=201
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
GET /api/contents/?content=1 HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Got a content-length, being smart about document end.
Calling initial content-length for extra data at end of headers
Spinning waiting for headers...
Calling after change function ‘url-http-content-length-after-change-function’ for ‘#<process localhost>’
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=200
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Found existing connection: localhost:8888 #<process localhost>
Reusing existing connection: localhost:8888
Marking connection as busy: localhost:8888 #<process localhost>
Request is: 
DELETE /api/contents/tramp.kN8HIA HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: localhost:8888
Accept-encoding: gzip
Accept: */*
User-Agent: jupyter/0.8.0-dev URL/Emacs Emacs/26.2 (nil; x86_64-apple-darwin18.5.0)
Cookie: _xsrf=2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977; username-localhost-8889=2|1:0|10:1560443415|23:username-localhost-8889|44:NTVkZmY0ZDhkMTBiNDg1YTkwNzg2NDgxNDJlNDBhNGI=|9ccff08cd8ea38b3245513397742025dbbf71115856ede597d2aa05e04f1806a; username-localhost-8888=2|1:0|10:1560525257|23:username-localhost-8888|44:MDcwNGNjMmYxMGQwNGI5M2I5NmM1MzVhMGJlNzQyYTE=|2df14ebc0d26af6301d197d3924fe0b774ae5074ca8f84add000f9d7c3b68bf3
X-XSRFTOKEN: 2|30ae629b|d5826abfb999e4eccc25effd62d133e3|1560442977

Spinning in url-retrieve-synchronously: nil (#<buffer  *http localhost:8888*>)
Calling after change function ‘url-http-wait-for-headers-change-function’ for ‘#<process localhost>’
url-http-wait-for-headers-change-function ( *http localhost:8888*)
Saw end of headers... ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
204 response must have headers only ( *http localhost:8888*).
Marking connection as free: localhost:8888 #<process localhost>
url-http-parse-headers called in ( *http localhost:8888*)
url-http-parse-response called in ( *http localhost:8888*)
Parsed HTTP headers: class=2 status=204
Finished parsing HTTP headers: t
Marking connection as free: localhost:8888 #<process localhost>
Activating callback in buffer ( *http localhost:8888*): #[128 "\302\303\304p#\210\300\305\240\210\301p\240\207" [(nil) (#<buffer  *http localhost:8888*>) url-debug retrieval "Synchronous fetching done (%S)" t] 5 "

(fn &rest IGNORED)"] (nil)
Synchronous fetching done (#<buffer  *http localhost:8888*>)
Spinning waiting for headers...

I then tried hitting RET on the kernel in the jupyter-server buffer and it failed to login.

Log from server:

[D 09:16:00.914 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/envs/dcpy/share/jupyter/kernels
[D 09:16:00.915 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 1.37ms
[D 09:16:00.929 NotebookApp] 200 GET /api/contents/?content=1 (127.0.0.1) 4.22ms
[I 09:16:00.946 NotebookApp] Uploading file to /tramp.kN8HIA
[D 09:16:00.946 NotebookApp] Saving /Users/dcherian/work/pump/notebooks/tramp.kN8HIA
[D 09:16:00.947 NotebookApp] 201 PUT /api/contents/tramp.kN8HIA (127.0.0.1) 1.56ms
[D 09:16:00.958 NotebookApp] 200 GET /api/contents/?content=1 (127.0.0.1) 3.01ms
[W 09:16:00.976 NotebookApp] delete /tramp.kN8HIA
[D 09:16:00.976 NotebookApp] Sending /Users/dcherian/work/pump/notebooks/tramp.kN8HIA to trash
[D 09:16:00.998 NotebookApp] 204 DELETE /api/contents/tramp.kN8HIA (127.0.0.1) 22.62ms
[D 09:17:49.514 NotebookApp] 200 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 0.78ms
[W 09:17:50.937 NotebookApp] Forbidden
[W 09:17:50.937 NotebookApp] 403 GET /api/kernelspecs (127.0.0.1) 1.01ms referer=None
[W 09:17:50.941 NotebookApp] 403 POST /login (127.0.0.1): '_xsrf' argument missing from POST
[D 09:17:50.942 NotebookApp] Using contents: services/contents
[D 09:17:50.942 NotebookApp] Using contents: services/contents
[D 09:17:50.963 NotebookApp] Path base/images/favicon.ico served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/base/images/favicon.ico
[D 09:17:50.964 NotebookApp] Path components/jquery-ui/themes/smoothness/jquery-ui.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
[D 09:17:50.964 NotebookApp] Path components/jquery-typeahead/dist/jquery.typeahead.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/jquery-typeahead/dist/jquery.typeahead.min.css
[D 09:17:50.965 NotebookApp] Path style/style.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/style/style.min.css
[D 09:17:50.966 NotebookApp] Path components/es6-promise/promise.min.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/es6-promise/promise.min.js
[D 09:17:50.967 NotebookApp] Path components/preact/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/preact/index.js
[D 09:17:50.967 NotebookApp] Path components/proptypes/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/proptypes/index.js
[D 09:17:50.968 NotebookApp] Path components/preact-compat/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/preact-compat/index.js
[D 09:17:50.968 NotebookApp] Path components/requirejs/require.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/requirejs/require.js
[D 09:17:50.969 NotebookApp] Path base/images/logo.png served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/base/images/logo.png
[W 09:17:50.970 NotebookApp] 403 POST /login (127.0.0.1) 29.24ms referer=None
[W 09:17:51.059 NotebookApp] Forbidden
[W 09:17:51.059 NotebookApp] 403 GET /api/kernelspecs (127.0.0.1) 0.61ms referer=None
[W 09:17:51.060 NotebookApp] Forbidden
[W 09:17:51.060 NotebookApp] 403 GET /api/kernelspecs (127.0.0.1) 0.41ms referer=None
[D 09:18:03.741 NotebookApp] 200 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 0.71ms
[D 09:18:03.746 NotebookApp] 200 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 0.58ms
[D 09:18:03.750 NotebookApp] 200 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 0.55ms
[D 09:18:03.754 NotebookApp] Clearing buffer for 4eab4943-fb7d-4f30-b517-4c0cfac340a5
[D 09:18:08.919 NotebookApp] Kernel is taking too long to finish, killing
[I 09:18:08.940 NotebookApp] Kernel shutdown: 4eab4943-fb7d-4f30-b517-4c0cfac340a5
[D 09:18:08.940 NotebookApp] 204 DELETE /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 5187.37ms
[W 09:18:57.280 NotebookApp] 404 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1): Kernel does not exist: 4eab4943-fb7d-4f30-b517-4c0cfac340a5
[W 09:18:57.280 NotebookApp] Kernel does not exist: 4eab4943-fb7d-4f30-b517-4c0cfac340a5
[W 09:18:57.280 NotebookApp] 404 GET /api/kernels/4eab4943-fb7d-4f30-b517-4c0cfac340a5 (127.0.0.1) 0.86ms referer=None

[dcherian]

Oh and I'm running Emacs 26.2 on MacOS mojave.

[nnicandro]

On Emacs 26.1.92 I've tested it with Jupyter 4.4.0 and Notebook 5.5.0, 5.7.8. I'm also on MacOS mojave. Everything seems to be working on my end though. I looked briefly at the Emacs 26.2 code and it doesn't look like anything has changed in the URL library.

Also looking at your HTTP requests, although not strictly necessary, each one of those requests should have the Authorization header. I wonder why they don't. When you have the kernel list buffer open, what does (oref jupyter-current-server auth) show?

Also the notebook log doesn't show the POST /api/kernels request that the url-debug log shows and none of the requests to the /login page that are shown in the notebook log are shown in the url-debug log. Can you try again with a fresh notebook. Also set url-debug to 1 before calling jupyter-server-list-kernels and show both the url-debug log and notebook log that correspond to all the requests. You probably want to start the notebook on a different port so that the notebook doesn't see the cookies that are now sent with each request made via the URL library, i.e. use the --NotebookApp.port setting. Alternatively you can delete the cookies by calling url-cookie-list and then url-cookie-delete on the lines containing the notebook cookies.

[nnicandro]

@dcherian Could you try again with the latest branch and see if you are able to connect successfully to the notebook server. I have fixed a few things that may have been related to the errors you were experiencing.

[dcherian]

Yes I can do this tomorrow morning (US). I'm currently flying back home.

[dcherian]

Now it seems to start a kernel but the jupyter-server-list-kernels buffer never updates status from starting.

[D 15:39:33.631 NotebookApp] Searching ['/Users/dcherian/work/pump/notebooks', '/Users/dcherian/.jupyter', '/Users/dcherian/anaconda3/envs/dcpy/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 15:39:33.632 NotebookApp] Looking for jupyter_config in /etc/jupyter
[D 15:39:33.632 NotebookApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 15:39:33.632 NotebookApp] Looking for jupyter_config in /Users/dcherian/anaconda3/envs/dcpy/etc/jupyter
[D 15:39:33.632 NotebookApp] Looking for jupyter_config in /Users/dcherian/.jupyter
[D 15:39:33.632 NotebookApp] Looking for jupyter_config in /Users/dcherian/work/pump/notebooks
[D 15:39:33.633 NotebookApp] Looking for jupyter_notebook_config in /etc/jupyter
[D 15:39:33.633 NotebookApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
[D 15:39:33.633 NotebookApp] Looking for jupyter_notebook_config in /Users/dcherian/anaconda3/envs/dcpy/etc/jupyter
[D 15:39:33.633 NotebookApp] Loaded config file: /Users/dcherian/anaconda3/envs/dcpy/etc/jupyter/jupyter_notebook_config.json
[D 15:39:33.634 NotebookApp] Looking for jupyter_notebook_config in /Users/dcherian/.jupyter
[D 15:39:33.634 NotebookApp] Looking for jupyter_notebook_config in /Users/dcherian/work/pump/notebooks
[D 15:39:34.378 NotebookApp] Paths used for configuration of jupyter_notebook_config:
        /etc/jupyter/jupyter_notebook_config.json
[D 15:39:34.378 NotebookApp] Paths used for configuration of jupyter_notebook_config:
        /usr/local/etc/jupyter/jupyter_notebook_config.json
[D 15:39:34.379 NotebookApp] Paths used for configuration of jupyter_notebook_config:
        /Users/dcherian/anaconda3/envs/dcpy/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
        /Users/dcherian/anaconda3/envs/dcpy/etc/jupyter/jupyter_notebook_config.json
[D 15:39:34.379 NotebookApp] Paths used for configuration of jupyter_notebook_config:
        /Users/dcherian/.jupyter/jupyter_notebook_config.json
[I 15:39:34.403 NotebookApp] JupyterLab extension loaded from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/jupyterlab
[I 15:39:34.403 NotebookApp] JupyterLab application directory is /Users/dcherian/anaconda3/envs/dcpy/share/jupyter/lab
[I 15:39:34.405 NotebookApp] Serving notebooks from local directory: /Users/dcherian/work/pump/notebooks
[I 15:39:34.405 NotebookApp] The Jupyter Notebook is running at:
[I 15:39:34.405 NotebookApp] http://localhost:8888/?token=fdd23e8db30413f7aa7026d31c78db514e3f41013ed7b58d
[I 15:39:34.405 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:39:34.409 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///Users/dcherian/Library/Jupyter/runtime/nbserver-62819-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=fdd23e8db30413f7aa7026d31c78db514e3f41013ed7b58d
[D 15:39:39.680 NotebookApp] 200 GET /api/kernels (127.0.0.1) 2.01ms
[D 15:39:41.536 NotebookApp] Starting kernel: ['/Users/dcherian/anaconda3/envs/dcpy/bin/python', '-m', 'ipykernel_launcher', '-f', '/Users/dcherian/Library/Jupyter/runtime/kernel-75645258-4bb4-4ddd-870f-f4b1d6444ee9.json']
[D 15:39:41.540 NotebookApp] Connecting to: tcp://127.0.0.1:59569
[D 15:39:41.541 NotebookApp] Connecting to: tcp://127.0.0.1:59567
[I 15:39:41.542 NotebookApp] Kernel started: 75645258-4bb4-4ddd-870f-f4b1d6444ee9
[D 15:39:41.543 NotebookApp] Kernel args: {'kernel_name': 'python3'}
[D 15:39:41.543 NotebookApp] 201 POST /api/kernels (127.0.0.1) 32.39ms
[D 15:39:41.549 NotebookApp] 200 GET /api/kernels (127.0.0.1) 1.07ms
[D 15:39:41.553 NotebookApp] 200 GET /api/kernels (127.0.0.1) 0.63ms
[D 15:39:43.537 NotebookApp] activity on 75645258-4bb4-4ddd-870f-f4b1d6444ee9: stream
[D 15:39:43.667 NotebookApp] activity on 75645258-4bb4-4ddd-870f-f4b1d6444ee9: status
[D 15:40:47.782 NotebookApp] 200 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9 (127.0.0.1) 1.05ms
[D 15:40:48.988 NotebookApp] Initializing websocket connection /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels
[W 15:40:48.990 NotebookApp] Couldn't authenticate WebSocket connection
[D 15:40:48.990 NotebookApp] Using contents: services/contents
[D 15:40:48.991 NotebookApp] Using contents: services/contents
[D 15:40:49.006 NotebookApp] Path base/images/favicon.ico served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/base/images/favicon.ico
[D 15:40:49.006 NotebookApp] Path components/jquery-ui/themes/smoothness/jquery-ui.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
[D 15:40:49.007 NotebookApp] Path components/jquery-typeahead/dist/jquery.typeahead.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/jquery-typeahead/dist/jquery.typeahead.min.css
[D 15:40:49.007 NotebookApp] Path style/style.min.css served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/style/style.min.css
[D 15:40:49.007 NotebookApp] Path components/es6-promise/promise.min.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/es6-promise/promise.min.js
[D 15:40:49.008 NotebookApp] Path components/preact/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/preact/index.js
[D 15:40:49.008 NotebookApp] Path components/proptypes/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/proptypes/index.js
[D 15:40:49.008 NotebookApp] Path components/preact-compat/index.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/preact-compat/index.js
[D 15:40:49.008 NotebookApp] Path components/requirejs/require.js served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/components/requirejs/require.js
[D 15:40:49.008 NotebookApp] Path base/images/logo.png served from /Users/dcherian/anaconda3/envs/dcpy/lib/python3.6/site-packages/notebook/static/base/images/logo.png
[W 15:40:49.009 NotebookApp] 403 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels?session_id=6c7bd6aa-bfed-48c1-bd75-65f037b97ca3 (127.0.0.1) 20.78ms referer=None
[D 15:40:49.012 NotebookApp] Initializing websocket connection /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels
[W 15:40:49.013 NotebookApp] Couldn't authenticate WebSocket connection
[D 15:40:49.013 NotebookApp] Using contents: services/contents
[D 15:40:49.013 NotebookApp] Using contents: services/contents
[W 15:40:49.014 NotebookApp] 403 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels?session_id=6c7bd6aa-bfed-48c1-bd75-65f037b97ca3 (127.0.0.1) 2.82ms referer=None
[D 15:40:52.204 NotebookApp] Initializing websocket connection /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels
[W 15:40:52.205 NotebookApp] Couldn't authenticate WebSocket connection
[D 15:40:52.206 NotebookApp] Using contents: services/contents
[D 15:40:52.206 NotebookApp] Using contents: services/contents
[W 15:40:52.207 NotebookApp] 403 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels?session_id=f28ecff2-2244-46ab-9d3f-7076b2b29407 (127.0.0.1) 2.67ms referer=None
[D 15:40:52.210 NotebookApp] Initializing websocket connection /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels
[W 15:40:52.211 NotebookApp] Couldn't authenticate WebSocket connection
[D 15:40:52.211 NotebookApp] Using contents: services/contents
[D 15:40:52.212 NotebookApp] Using contents: services/contents
[W 15:40:52.212 NotebookApp] 403 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels?session_id=f28ecff2-2244-46ab-9d3f-7076b2b29407 (127.0.0.1) 2.33ms referer=None
[D 15:41:56.571 NotebookApp] 200 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9 (127.0.0.1) 0.84ms
[D 15:41:56.576 NotebookApp] 200 GET /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9 (127.0.0.1) 0.70ms
[D 15:41:56.579 NotebookApp] Clearing buffer for 75645258-4bb4-4ddd-870f-f4b1d6444ee9
[D 15:42:01.741 NotebookApp] Kernel is taking too long to finish, killing
[I 15:42:01.761 NotebookApp] Kernel shutdown: 75645258-4bb4-4ddd-870f-f4b1d6444ee9
[D 15:42:01.761 NotebookApp] 204 DELETE /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9 (127.0.0.1) 5182.40ms
[D 15:42:21.156 NotebookApp] Found kernel python3 in /Users/dcherian/anaconda3/envs/dcpy/share/jupyter/kernels
[D 15:42:21.157 NotebookApp] 200 GET /api/kernelspecs (127.0.0.1) 1.48ms
[D 15:42:21.163 NotebookApp] 200 GET /api/contents?content=1 (127.0.0.1) 3.94ms
[I 15:42:21.578 NotebookApp] Uploading file to /tramp.U4ECTV
[D 15:42:21.578 NotebookApp] Saving /Users/dcherian/work/pump/notebooks/tramp.U4ECTV
[D 15:42:21.579 NotebookApp] 201 PUT /api/contents/tramp.U4ECTV (127.0.0.1) 1.74ms
[D 15:42:21.589 NotebookApp] 200 GET /api/contents?content=1 (127.0.0.1) 3.84ms
[W 15:42:21.598 NotebookApp] delete /tramp.U4ECTV
[D 15:42:21.598 NotebookApp] Sending /Users/dcherian/work/pump/notebooks/tramp.U4ECTV to trash
[D 15:42:21.629 NotebookApp] 204 DELETE /api/contents/tramp.U4ECTV (127.0.0.1) 32.24ms
[D 15:42:21.748 NotebookApp] 200 GET /api/contents?content=1 (127.0.0.1) 4.92ms

EDIT: looks like kernel eventually gets killed.

[dcherian]

Hitting C-c C-c on a code block with header #+BEGIN_SRC jupyter-python :session /jpy:localhost#8888/ actually runs things but I'm not sure that it's actually running on the notebook server

[nnicandro]

Looks like there is something going wrong trying to authenticate the websocket since there are a lot of lines like

[D 15:40:52.210 NotebookApp] Initializing websocket connection /api/kernels/75645258-4bb4-4ddd-870f-f4b1d6444ee9/channels
[W 15:40:52.211 NotebookApp] Couldn't authenticate WebSocket connection

Can you show the value of the websocket-version variable. Also, does url-cookie-file point to an actual file? After you try to connect do you see any cookies after evaluating M-x url-cookie-list that look like

localhost        _xsrf                2|c7d1c2e8|b66c417fd8d2aecead6e9f5290330d45|1561310292
localhost:49446  _xsrf                2|c7d1c2e8|b66c417fd8d2aecead6e9f5290330d45|1561310292

the second one with the port is important for the websocket code.

For the org-babel code block, the session should be /jpy:localhost#8888:NAME (notice the second :) and where NAME is the name of the session (or it can be an existing kernel ID). /jpy:localhost#8888/ is not a valid remote file name so that was just a local kernel that started up with the session name being /jpy:localhost#8888/.

[dcherian]

websocket version is 1.11.1 and url-cookie-file is an actual file.

I am still getting websocket authentication errors and url-cookie-list looks like this

localhost username-localhost-8 2|1:0|10:1562360223|23:username-localhost-8888|44:ZDE1ZWE5OTVlMTIwNDU3ZDg1ZjEwMzE1ODFmMmRjMjk=|ab8d1b243034e770fb27066812054d9172e74f996240e8848614d8b101606606

[nnicandro]

Ok you should have the _xsrf cookie being shown in url-cookie-list. The fact that it doesn't appear did lead me to an issue which I fixed in 99fd8fe3c4430664badc78d750b4717327e83d20. Can you try again to see if it works this time.

Also can you confirm whether or not you are being asked for a token every time you try? The username-localhost-8888 cookie being present means that you were probably asked for a token on a previous login attempt which succeeded (and set the login cookie), but on trying again the server saw the login cookie and considered the connection already authenticated. If you were not asked for the token on your most recent attempt to connect a websocket, 99fd8fe3c4430664badc78d750b4717327e83d20 should fix the issue.

Also for reference, whenever a token has been asked for and is being used for authentication you should see messages like the following in the Jupyter notebook log

[D 15:00:18.321 NotebookApp] Accepting token-authenticated connection from ::1
[D 15:00:18.325 NotebookApp] 101 GET /api/kernels/fe468bc0-01a9-4715-966f-1759137497ac/channels?session_id=36f4a882-8a32-4dff-8053-fe03e8c6c5cf (::1) 10.84ms

[dcherian]

OK I successfully connected on the first try. It didn't ask for a token but I have a REPL. The only cookie is

localhost _xsrf                2|d3adbc8b|f2e9102b2557e380bb17ba09af2d9509|1562459168

I'm getting jupyter-read-plist-from-string: Wrong type argument: char-or-string-p, nil when I try to execute a code block though.

PS: Thanks for all the work!

[dcherian]

OK I can dependably close a kernel too from jupyter-server-list-kernels.

WHen starting using C-RET, it adds a kernel entry but the state just says "Starting". When I hit RET on the server, it prints Requesting kernel info and then successfully starts a REPL.

[D 18:33:28.969 NotebookApp] 204 DELETE /api/kernels/73c43f76-3594-48f3-8418-22d8ac6c4c3d (::1) 1307.56ms
[D 18:33:34.904 NotebookApp] Starting kernel: ['/home/deepak/miniconda3/envs/dcpy/bin/python', '-m', 'ipykernel_launcher', '-f', '/run/user/1000/jupyter/kernel-14683651-3a67-42ba-9750-cd10952d4c85.json']
[D 18:33:34.913 NotebookApp] Connecting to: tcp://127.0.0.1:33593
[D 18:33:34.915 NotebookApp] Connecting to: tcp://127.0.0.1:50713
[I 18:33:34.918 NotebookApp] Kernel started: 14683651-3a67-42ba-9750-cd10952d4c85
[D 18:33:34.918 NotebookApp] Kernel args: {'kernel_name': 'python3'}
[D 18:33:34.920 NotebookApp] 201 POST /api/kernels (::1) 23.49ms
[D 18:33:34.926 NotebookApp] 200 GET /api/kernels (::1) 2.48ms
[D 18:33:36.177 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
QApplication: invalid style override passed, ignoring it.
[W 18:34:50.624 NotebookApp] 404 GET /api/kernels/4a03c4c0-8d3c-4299-8968-350629d347fe (::1): Kernel does not exist: 4a03c4c0-8d3c-4299-8968-350629d347fe
[W 18:34:50.624 NotebookApp] Kernel does not exist: 4a03c4c0-8d3c-4299-8968-350629d347fe
[W 18:34:50.625 NotebookApp] 404 GET /api/kernels/4a03c4c0-8d3c-4299-8968-350629d347fe (::1) 2.21ms referer=None
[W 18:34:50.628 NotebookApp] 404 GET /api/kernels/73c43f76-3594-48f3-8418-22d8ac6c4c3d (::1): Kernel does not exist: 73c43f76-3594-48f3-8418-22d8ac6c4c3d
[W 18:34:50.629 NotebookApp] Kernel does not exist: 73c43f76-3594-48f3-8418-22d8ac6c4c3d
[W 18:34:50.629 NotebookApp] 404 GET /api/kernels/73c43f76-3594-48f3-8418-22d8ac6c4c3d (::1) 1.93ms referer=None
[D 18:34:50.635 NotebookApp] 200 GET /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85 (::1) 1.44ms
[D 18:34:50.669 NotebookApp] Initializing websocket connection /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85/channels
[D 18:34:50.673 NotebookApp] Requesting kernel info from 14683651-3a67-42ba-9750-cd10952d4c85
[D 18:34:50.673 NotebookApp] Connecting to: tcp://127.0.0.1:51609
[D 18:34:50.681 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:34:50.682 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:34:50.684 NotebookApp] Received kernel info: {'status': 'ok', 'protocol_version': '5.1', 'implementation': 'ipython', 'implementation_version': '7.6.1', 'language_info': {'name': 'python', 'version': '3.7.3', 'mimetype': 'text/x-python', 'codemirror_mode': {'name': 'ipython', 'version': 3}, 'pygments_lexer': 'ipython3', 'nbconvert_exporter': 'python', 'file_extension': '.py'}, 'banner': "Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21) \nType 'copyright', 'credits' or 'license' for more information\nIPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.\n", 'help_links': [{'text': 'Python Reference', 'url': 'https://docs.python.org/3.7'}, {'text': 'IPython Reference', 'url': 'https://ipython.org/documentation.html'}, {'text': 'NumPy Reference', 'url': 'https://docs.scipy.org/doc/numpy/reference/'}, {'text': 'SciPy Reference', 'url': 'https://docs.scipy.org/doc/scipy/reference/'}, {'text': 'Matplotlib Reference', 'url': 'https://matplotlib.org/contents.html'}, {'text': 'SymPy Reference', 'url': 'http://docs.sympy.org/latest/index.html'}, {'text': 'pandas Reference', 'url': 'https://pandas.pydata.org/pandas-docs/stable/'}]}
[I 18:34:50.684 NotebookApp] Adapting to protocol v5.1 for kernel 14683651-3a67-42ba-9750-cd10952d4c85
[D 18:34:50.685 NotebookApp] 101 GET /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85/channels?session_id=c82ba8bc-9828-48ef-bd72-486ed66a8f46 (::1) 16.58ms
[D 18:34:50.686 NotebookApp] Opening websocket /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85/channels
[D 18:34:50.686 NotebookApp] Getting buffer for 14683651-3a67-42ba-9750-cd10952d4c85
[D 18:34:50.686 NotebookApp] Connecting to: tcp://127.0.0.1:51609
[D 18:34:50.687 NotebookApp] Connecting to: tcp://127.0.0.1:50713
[D 18:34:50.687 NotebookApp] Connecting to: tcp://127.0.0.1:39897
[D 18:34:51.684 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:34:51.690 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:34:52.682 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:34:52.685 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:01.741 NotebookApp] 200 GET /api/kernels (::1) 1.60ms
[D 18:35:19.478 NotebookApp] 200 GET /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85 (::1) 1.63ms
[D 18:35:19.564 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:19.571 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:20.566 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:20.572 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:21.569 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:21.574 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:23.645 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:24.101 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:25.946 NotebookApp] 200 GET /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85 (::1) 1.66ms
[D 18:35:26.132 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:26.140 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:26.220 NotebookApp] 200 GET /api/kernels/14683651-3a67-42ba-9750-cd10952d4c85 (::1) 1.57ms
[D 18:35:27.131 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: status
[D 18:35:27.133 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: execute_input
[D 18:35:27.176 NotebookApp] activity on 14683651-3a67-42ba-9750-cd10952d4c85: execute_result

EDIT:

I'm getting jupyter-read-plist-from-string: Wrong type argument: char-or-string-p, nil when I try to execute a code block though.

Here's a trace

Debugger entered--Lisp error: (wrong-type-argument char-or-string-p nil)
  json-read-from-string(nil)
  jupyter-read-plist-from-string(nil)
  jupyter-available-kernelspecs(nil)
  jupyter-find-kernelspecs("python")
  jupyter-run-repl("python" nil nil jupyter-org-client)
  org-babel-jupyter--run-repl("/jpy:localhost#8888:14683651-3a67-42ba-9750-cd1095..." "python")
  org-babel-jupyter-initiate-session-by-key("/jpy:localhost#8888:14683651-3a67-42ba-9750-cd1095..." ((:results . "replace") (:exports . "code") (:session . "/jpy:localhost#8888:14683651-3a67-42ba-9750-cd1095...") (:async . "no") (:kernel . "python") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")))
  org-babel-jupyter-initiate-session("/jpy:localhost#8888:14683651-3a67-42ba-9750-cd1095..." ((:results . "replace") (:exports . "code") (:session . "/jpy:localhost#8888:14683651-3a67-42ba-9750-cd1095...") (:async . "no") (:kernel . "python") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")))
  org-babel-initiate-session(nil nil)
  org-babel-switch-to-session(nil)
  funcall-interactively(org-babel-switch-to-session nil)
  call-interactively(org-babel-switch-to-session nil nil)
  command-execute(org-babel-switch-to-session)

[nnicandro]

org-babel-jupyter--run-repl shouldn't be getting called, maybe you have stale .elc files? To ensure you are using the updated org babel functions when you don't want to restart Emacs, call M-x eval-buffer on the ob-jupyter.el file. Alternatively you can M-x byte-recompile-directory in the project directory and restart Emacs.

You may also need to evaluate (require 'jupyter-tramp) if the variable tramp-methods doesn't show an entry like

 ("jpy"
  (tramp-default-port 8888)
  (tramp-tmpdir "/tmp"))

[dcherian]

call M-x eval-buffer on the ob-jupyter.el file.

This still gives me the same error

Debugger entered--Lisp error: (wrong-type-argument char-or-string-p nil)
  json-read-from-string(nil)
  jupyter-read-plist-from-string(nil)
  jupyter-available-kernelspecs(nil)
  (let* ((--cl-var-- (jupyter-available-kernelspecs refresh)) (spec nil) (kernel nil) (--cl-var--) (lang nil) (languages nil)) (while (consp --cl-var--) (setq spec (car --cl-var--) kernel (car-safe (prog1 spec (setq spec (cdr spec)))) --cl-var-- (car-safe (prog1 spec (setq spec (cdr spec))))) (setq lang (plist-get spec :language)) (if (member lang languages) (progn) (setq languages (nconc languages (list lang))) (org-babel-jupyter-make-language-alias kernel lang) (if (assoc lang org-babel-tangle-lang-exts) (progn (add-to-list 'org-babel-tangle-lang-exts (cons (concat "jupyter-" lang) (cdr ...))))) (add-to-list 'org-src-lang-modes (cons (concat "jupyter-" lang) (or (cdr (assoc lang org-src-lang-modes)) (intern (downcase ...)))))) (setq --cl-var-- (cdr --cl-var--))) nil)
  org-babel-jupyter-aliases-from-kernelspecs()
  eval-buffer()  ; Reading at buffer position 25214
  funcall-interactively(eval-buffer)
  call-interactively(eval-buffer nil nil)
  command-execute(eval-buffer)

However things do work! I can start a REPL and send code from org mode blocks. I have to use #+BEGIN_SRC jupyter instead of #+BEGIN_SRC jupyter-python so something is screwed up. The error I get is org-babel-execute-src-block: No org-babel-execute function for jupyter-python!

[nnicandro]

Do you get that same error if you M-x byte-recompile-directory in the project directory, restart Emacs, and try to connect to the notebook kernel again? Note, you will probably have to evaluate (require 'jupyter-tramp).

[dcherian]

byte-recompile-directory just skips all files. I have deleted the directory under elpa and reinstalled from file.

Debugger entered--Lisp error: (wrong-type-argument char-or-string-p nil)
  json-read-from-string(nil)
  jupyter-read-plist-from-string(nil)
  jupyter-available-kernelspecs(nil)
  org-babel-jupyter-aliases-from-kernelspecs()
  byte-code("\300 \210\301\302\303\"\210\301\304\305\"\210\306\307!\207" [org-babel-jupyter-aliases-from-kernelspecs add-hook org-export-before-processing-hook org-babel-jupyter-setup-export org-export-before-parsing-hook org-babel-jupyter-strip-ansi-escapes provide ob-jupyter] 3)
  require(ob-jupyter)
  org-babel-do-load-languages(org-babel-load-languages ((matlab . t) (emacs-lisp . t) (latex . t) (shell . t) (python . t) (jupyter . t)))

I still get this error which seems to the be the root problem. M-x eval-buffer on ob-jupyter.el fails with the same error

[nnicandro]

When you run (shell-command "jupyter kernelspec list --json") does it return valid JSON or does it produce an error? Most likely it is producing an error if nil is being passed to jupyter-read-plist-from-string.

This error also brings up another issue. We shouldn't be relying only on the jupyter shell command to create language aliases for org-mode anymore. Since we now support the REST API, the kernelspecs we get through the jupyter shell command may be missing kernels that a user can access through a notebook server.

[dcherian]

Apologies! I was playing around with virtual environments and didn't have the right one activated. It would be nice to raise a more informative error. (ob-ipython had this issue where it kept raising errors because it wasn't receiving JSON but it was really hard to inspect and debug.)

Everything works well now. I can start new kernels and connect to existing ones. This is awesome! Thanks!

The only issue I see now is when starting a new kernel with C-RET in the jupyter-kernels buffer. The state just says Starting.

[D 09:53:07.780 NotebookApp] Starting kernel: ['/home/deepak/miniconda3/envs/dcpy/bin/python', '-m', 'ipykernel_launcher', '-f', '/run/user/1000/jupyter/kernel-7dd3caff-1d68-4f15-bb73-b5a554bddb77.json']
[D 09:53:07.792 NotebookApp] Connecting to: tcp://127.0.0.1:41771
[D 09:53:07.794 NotebookApp] Connecting to: tcp://127.0.0.1:50975
[I 09:53:07.798 NotebookApp] Kernel started: 7dd3caff-1d68-4f15-bb73-b5a554bddb77
[D 09:53:07.798 NotebookApp] Kernel args: {'kernel_name': 'python3'}
[D 09:53:07.800 NotebookApp] 201 POST /api/kernels (::1) 27.92ms
[D 09:53:07.808 NotebookApp] 200 GET /api/kernels (::1) 3.03ms
[D 09:53:09.050 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
QApplication: invalid style override passed, ignoring it.

Hitting RET on this opens up a kenel properly though. Then I see:

[W 09:54:48.536 NotebookApp] 404 GET /api/kernels/083d3043-5ec9-4831-ade5-59298199bcf8 (::1): Kernel does not exist: 083d3043-5ec9-4831-ade5-59298199bcf8
[W 09:54:48.536 NotebookApp] Kernel does not exist: 083d3043-5ec9-4831-ade5-59298199bcf8
[W 09:54:48.537 NotebookApp] 404 GET /api/kernels/083d3043-5ec9-4831-ade5-59298199bcf8 (::1) 2.52ms referer=None
[W 09:54:48.541 NotebookApp] 404 GET /api/kernels/ca7c7578-8b31-4bd8-a97d-b87ea3fa88c6 (::1): Kernel does not exist: ca7c7578-8b31-4bd8-a97d-b87ea3fa88c6
[W 09:54:48.542 NotebookApp] Kernel does not exist: ca7c7578-8b31-4bd8-a97d-b87ea3fa88c6
[W 09:54:48.543 NotebookApp] 404 GET /api/kernels/ca7c7578-8b31-4bd8-a97d-b87ea3fa88c6 (::1) 2.02ms referer=None
[W 09:54:48.547 NotebookApp] 404 GET /api/kernels/a4c58d1b-6ada-4351-9413-bc9074b3d4f8 (::1): Kernel does not exist: a4c58d1b-6ada-4351-9413-bc9074b3d4f8
[W 09:54:48.547 NotebookApp] Kernel does not exist: a4c58d1b-6ada-4351-9413-bc9074b3d4f8
[W 09:54:48.548 NotebookApp] 404 GET /api/kernels/a4c58d1b-6ada-4351-9413-bc9074b3d4f8 (::1) 1.94ms referer=None
[D 09:54:48.552 NotebookApp] 200 GET /api/kernels/7dd3caff-1d68-4f15-bb73-b5a554bddb77 (::1) 1.51ms
[D 09:54:48.648 NotebookApp] Initializing websocket connection /api/kernels/7dd3caff-1d68-4f15-bb73-b5a554bddb77/channels
[D 09:54:48.651 NotebookApp] Requesting kernel info from 7dd3caff-1d68-4f15-bb73-b5a554bddb77
[D 09:54:48.651 NotebookApp] Connecting to: tcp://127.0.0.1:33857
[D 09:54:48.659 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:48.660 NotebookApp] Received kernel info: {'status': 'ok', 'protocol_version': '5.1', 'implementation': 'ipython', 'implementation_version': '7.6.1', 'language_info': {'name': 'python', 'version': '3.7.3', 'mimetype': 'text/x-python', 'codemirror_mode': {'name': 'ipython', 'version': 3}, 'pygments_lexer': 'ipython3', 'nbconvert_exporter': 'python', 'file_extension': '.py'}, 'banner': "Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21) \nType 'copyright', 'credits' or 'license' for more information\nIPython 7.6.1 -- An enhanced Interactive Python. Type '?' for help.\n", 'help_links': [{'text': 'Python Reference', 'url': 'https://docs.python.org/3.7'}, {'text': 'IPython Reference', 'url': 'https://ipython.org/documentation.html'}, {'text': 'NumPy Reference', 'url': 'https://docs.scipy.org/doc/numpy/reference/'}, {'text': 'SciPy Reference', 'url': 'https://docs.scipy.org/doc/scipy/reference/'}, {'text': 'Matplotlib Reference', 'url': 'https://matplotlib.org/contents.html'}, {'text': 'SymPy Reference', 'url': 'http://docs.sympy.org/latest/index.html'}, {'text': 'pandas Reference', 'url': 'https://pandas.pydata.org/pandas-docs/stable/'}]}
[I 09:54:48.660 NotebookApp] Adapting to protocol v5.1 for kernel 7dd3caff-1d68-4f15-bb73-b5a554bddb77
[D 09:54:48.662 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:48.663 NotebookApp] 101 GET /api/kernels/7dd3caff-1d68-4f15-bb73-b5a554bddb77/channels?session_id=6e2ffb90-cd9b-4456-8ca9-45cbe141901d (::1) 16.22ms
[D 09:54:48.664 NotebookApp] Opening websocket /api/kernels/7dd3caff-1d68-4f15-bb73-b5a554bddb77/channels
[D 09:54:48.664 NotebookApp] Getting buffer for 7dd3caff-1d68-4f15-bb73-b5a554bddb77
[D 09:54:48.664 NotebookApp] Connecting to: tcp://127.0.0.1:33857
[D 09:54:48.665 NotebookApp] Connecting to: tcp://127.0.0.1:50975
[D 09:54:48.666 NotebookApp] Connecting to: tcp://127.0.0.1:51815
[D 09:54:49.661 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:49.663 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:50.664 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:50.674 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:59.718 NotebookApp] 200 GET /api/kernels (::1) 1.76ms
[D 09:54:59.748 NotebookApp] 200 GET /api/kernels/7dd3caff-1d68-4f15-bb73-b5a554bddb77 (::1) 1.93ms
[D 09:54:59.910 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:54:59.917 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:55:00.907 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status
[D 09:55:00.908 NotebookApp] activity on 7dd3caff-1d68-4f15-bb73-b5a554bddb77: status

[nnicandro]

raise a more informative error

👍

The only issue I see now is when starting a new kernel with C-RET in the jupyter-kernels buffer. The state just says Starting.

The state of the kernel is received from the notebook REST API so we can't do much about that. For some reason the notebook keeps the state of the kernel as starting until a client connects to it. Presumably until a status: idle message is sent by the kernel to a client.

[dcherian]

A usability question: Is there an easy way to associate a notebook kernel with the buffer? Or specify a name when creating a kernel? It's a little annoying to keep changing the :session argument in code blocks...

[nnicandro]

If you specify :session with some name that does not correspond to an existing kernel, a new kernel will be created on the notebook server and the name associated with it. So if you specify :session /jpy::jaynes-ITSM a kernel will be launched on the notebook server and you can instead pass around /jpy::jaynes-ITSM as the :session instead of a kernel ID.

The above solution is only a side effect of the current implementation. I would like to be able to have names associated to kernels that is also visible in the kernel list. Its not very useful to have a bunch of kernels with names like julia, julia<1>, ... It would also be useful to have name associations persist across Emacs sessions for long running notebook servers.

[dcherian]

OK here's an issue. I'm connected to a kernel that is tunneled through on localhost:8800.

I managed to change the server URL (what is the best way to do this? C-u jupyter-server-list-kernels?)

I can start kernels properly with C-RET (confirmed with remote jupyter server log) but can't connect a REPL because it's trying to connect over 8888 instead of 8800

open-network-stream: make client process failed: Connection refused, :name, localhost, :buffer, #<killed buffer>, :host, localhost, :service, 8888, :nowait, nil, :tls-parameters, nilInvalid face attribute :foreground nil

EDIT: C-c C-v C-z on a source block works and opens a REPL

[dcherian]

If you specify :session with some name that does not correspond to an existing kernel, a new kernel will be created on the notebook server and the name associated with it. So if you specify :session /jpy::jaynes-ITSM a kernel will be launched on the notebook server and you can instead pass around /jpy::jaynes-ITSM as the :session instead of a kernel ID.

How do I associate a notebook server with the current buffer?

The above solution is only a side effect of the current implementation. I would like to be able to have names associated to kernels that is also visible in the kernel list. Its not very useful to have a bunch of kernels with names like julia, julia<1>, ... It would also be useful to have name associations persist across Emacs sessions for long running notebook servers.

Ah great.

[nnicandro]

How do I associate a notebook server with the current buffer?

Can you do something like in https://github.com/dzop/emacs-jupyter/issues/136#issue-459598396?

[nnicandro]

I managed to change the server URL (what is the best way to do this? C-u jupyter-server-list-kernels?)

Yes C-u jupyter-server-list-kernels is the way to do it. I should probably add that to the documentation. Commands that work with a kernel server usually use the jupyter-current-server which can be changed by providing a C-u prefix to such commands. The selected server then becomes the jupyter-current-server and subsequent commands will use it until a different server is selected.

trying to connect over 8888 instead of 8800

Can you show a backtrace of the error?

[dcherian]

It says localhost:8800 on top so this was really surprising.

Debugger entered--Lisp error: (file-error "make client process failed" "Connection refused" :name "localhost" :buffer #<buffer  *url-http-temp*> :host "localhost" :service 8888 :nowait nil :tls-parameters nil)
  make-network-process(:name "localhost" :buffer #<buffer  *url-http-temp*> :host "localhost" :service 8888 :nowait nil :tls-parameters nil)
  open-network-stream("localhost" #<buffer  *url-http-temp*> "localhost" 8888 :type plain :nowait nil)
  url-open-stream("localhost" #<buffer  *url-http-temp*> "localhost" 8888 nil)
  url-http-find-free-connection("localhost" 8888 nil)
  url-http(#s(url :type "http" :user nil :password nil :host "localhost" :portspec 8888 :filename "/api/kernels/083d3043-5ec9-4831-ade5-59298199bcf8" :target nil :attributes nil :fullness t :silent t :use-cookies t :asynchronous nil) #f(compiled-function (&rest ignored) #<bytecode 0x1c49aa9>) (nil))
  url-retrieve-internal("http://localhost:8888/api/kernels/083d3043-5ec9-48..." #f(compiled-function (&rest ignored) #<bytecode 0x1c49aa9>) (nil) t nil)
  url-retrieve("http://localhost:8888/api/kernels/083d3043-5ec9-48..." #f(compiled-function (&rest ignored) #<bytecode 0x1c49aa9>) nil t nil)
  url-retrieve-synchronously("http://localhost:8888/api/kernels/083d3043-5ec9-48..." t nil 10)
  jupyter-api-url-request("http://localhost:8888/api/kernels/083d3043-5ec9-48...")
  jupyter-api-http-request("http://localhost:8888" "api/kernels/083d3043-5ec9-4831-ade5-59298199bcf8" "GET")
  apply(jupyter-api-http-request "http://localhost:8888" "api/kernels/083d3043-5ec9-4831-ade5-59298199bcf8" "GET" nil)
  #f(compiled-function (client method &rest plist) "Send an HTTP request using CLIENT.\nMETHOD is the HTTP request method and PLIST contains the request.\nThe elements of PLIST before the first non-string form the REST\nAPI endpoint and the rest of the PLIST after will be encoded into\na JSON object and sent as the request data. So a call like\n\n   (jupyter-api-request client \"POST\" \"api\" \"kernels\" :name \"python\")\n\nwhere the url slot of client is http://localhost:8888 will create\nan http POST request to the url http://localhost:8888/api/kernels\nusing the JSON encoded from the plist (:name \"python\") as the\nPOST data.\n\nNote an empty plist (after forming the endpoint) is interpreted\nas no request data at all and NOT as an empty JSON dictionary.\n\nA call to this method can also look like\n\n   (jupyter-api-request client \"GET\"\n      \"api\" \"contents\" '((\"content\" . \"1\"))\n\nIn this case, the alist after the strings that make up the base\nendpoint, but before the rest of the non-strings elements of\nPLIST, will be interpreted as the query component of the\nresulting endpoint. So for the above example, the resulting url\nwill be http://localhost:8888/api/contents?content=1.\n\nIf METHOD is \"WS\", a websocket will be opened using the REST api\nurl and PLIST will be used in a call to `websocket-open'." #<bytecode 0x2b129b5>)(#<jupyter-server jupyter-server-9bba68> "GET" "api" "kernels" "083d3043-5ec9-4831-ade5-59298199bcf8")
  apply(#f(compiled-function (client method &rest plist) "Send an HTTP request using CLIENT.\nMETHOD is the HTTP request method and PLIST contains the request.\nThe elements of PLIST before the first non-string form the REST\nAPI endpoint and the rest of the PLIST after will be encoded into\na JSON object and sent as the request data. So a call like\n\n   (jupyter-api-request client \"POST\" \"api\" \"kernels\" :name \"python\")\n\nwhere the url slot of client is http://localhost:8888 will create\nan http POST request to the url http://localhost:8888/api/kernels\nusing the JSON encoded from the plist (:name \"python\") as the\nPOST data.\n\nNote an empty plist (after forming the endpoint) is interpreted\nas no request data at all and NOT as an empty JSON dictionary.\n\nA call to this method can also look like\n\n   (jupyter-api-request client \"GET\"\n      \"api\" \"contents\" '((\"content\" . \"1\"))\n\nIn this case, the alist after the strings that make up the base\nendpoint, but before the rest of the non-strings elements of\nPLIST, will be interpreted as the query component of the\nresulting endpoint. So for the above example, the resulting url\nwill be http://localhost:8888/api/contents?content=1.\n\nIf METHOD is \"WS\", a websocket will be opened using the REST api\nurl and PLIST will be used in a call to `websocket-open'." #<bytecode 0x2b129b5>) #<jupyter-server jupyter-server-9bba68> ("GET" "api" "kernels" "083d3043-5ec9-4831-ade5-59298199bcf8"))
  jupyter-api-request(#<jupyter-server jupyter-server-9bba68> "GET" "api" "kernels" "083d3043-5ec9-4831-ade5-59298199bcf8")
  apply(jupyter-api-request #<jupyter-server jupyter-server-9bba68> "GET" "api" "kernels" "083d3043-5ec9-4831-ade5-59298199bcf8")
  #f(compiled-function (client method &rest plist) "Send an HTTP request to the api/kernels endpoint to CLIENT's url.\nMETHOD is the HTTP method to use. PLIST has the same meaning as\nin `jupyter-api-request'." #<bytecode 0x1585825>)(#<jupyter-server jupyter-server-9bba68> "GET" "083d3043-5ec9-4831-ade5-59298199bcf8")
  apply(#f(compiled-function (client method &rest plist) "Send an HTTP request to the api/kernels endpoint to CLIENT's url.\nMETHOD is the HTTP method to use. PLIST has the same meaning as\nin `jupyter-api-request'." #<bytecode 0x1585825>) #<jupyter-server jupyter-server-9bba68> ("GET" "083d3043-5ec9-4831-ade5-59298199bcf8"))
  jupyter-api/kernels(#<jupyter-server jupyter-server-9bba68> "GET" "083d3043-5ec9-4831-ade5-59298199bcf8")
  jupyter-api-get-kernel(#<jupyter-server jupyter-server-9bba68> "083d3043-5ec9-4831-ade5-59298199bcf8")
  #f(compiled-function (kernel) #<bytecode 0x20fa1dd>)(#<jupyter-server-kernel jupyter-server-kernel-1073190>)
  apply(#f(compiled-function (kernel) #<bytecode 0x20fa1dd>) #<jupyter-server-kernel jupyter-server-kernel-1073190> nil)
  jupyter-kernel-alive-p(#<jupyter-server-kernel jupyter-server-kernel-1073190>)
  #f(compiled-function (manager) #<bytecode 0x1ab85dd>)(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1073544>)
  apply(#f(compiled-function (manager) #<bytecode 0x1ab85dd>) #<jupyter-server-kernel-manager jupyter-server-kernel-manager-1073544> nil)
  jupyter-kernel-alive-p(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1073544>)
  jupyter-server-find-manager(#<jupyter-server jupyter-server-1ab23dc> "ff958af2-5ec1-49a9-ac98-dd424157d448")
  jupyter-connect-server-repl(#<jupyter-server jupyter-server-1ab23dc> "ff958af2-5ec1-49a9-ac98-dd424157d448")
  jupyter-server-kernel-list-new-repl()
  funcall-interactively(jupyter-server-kernel-list-new-repl)
  call-interactively(jupyter-server-kernel-list-new-repl nil nil)
  command-execute(jupyter-server-kernel-list-new-repl)

[nnicandro]

Thanks I found the problem. Should be fixed by 7324c3174c64218bd9fd5a24aa276c46c23cf9cb and 14f199b10891d74def2ab8bd4f015793436c9179.

[dcherian]

Yesterday I killed all kernels and closed my ssh session. At least I think I killed the kernels but this may not have been cleanly closed.

Today I am running into 403 Acess Forbidden errors trying to connect over the same port (8888). I deleted cookies in url-cookie-list but that doesn't seem to help.

a) How do I fix this? b) Maybe we need a jupyter-cleanup-all-server-connections like tramp?

[dcherian]

My workaround for the latest problem is to tunnel on a different port and change jupyter-current-server

Thanks I found the problem. Should be fixed by 7324c31 and 14f199b.

Confirmed fixed. Thanks

[nnicandro]

I am running into 403 Acess Forbidden errors trying to connect over the same port (8888).

Can you show the backtrace of these errors.

Are you connecting to the same notebook process or is it different? If the latter it could be that the token changed. I used to have it so that if these Access Forbidden errors happened, I would ask the user to re-authenticate the connection. It seems like it would be a good idea to bring this back since in these kinds of cases there isn't really an easy way to get back to a usable state other than deleting the server object.

jupyter-cleanup-all-server-connections

There already is jupyter-gc-servers, but it currently only deletes the server objects that are not accessible at their ports. Does it make sense to also delete the server objects if they give Access Forbidden errors or should we just try to have the user re-authenticate?

Either way would work, but I think attempting to re-authenticate would be simpler since deleting the server object means we would have to do something about any buffers or clients using that server object.

[dcherian]

Are you connecting to the same notebook process or is it different? If the latter it could be that the token changed.

A different process so it makes sense that it can't authenticate. I just didn't know how to tell it to forget about that server. Deleting the cookie didn't help.

I used to have it so that if these Access Forbidden errors happened, I would ask the user to re-authenticate the connection.

This is a good idea for this case.

Does it make sense to also delete the server objects if they give Access Forbidden errors or should we just try to have the user re-authenticate?

I think reauthenticate. jupyter-gc-servers seems good but isn't interactive? That could be the backup-nuke-everything-and-start-again option.

I get this traceback on eval-ing (jupyter-gc-servers)

Debugger entered--Lisp error: (unbound-slot jupyter-server "#<jupyter-server jupyter-server-120c710>" ioloop oref)
  signal(unbound-slot (jupyter-server "#<jupyter-server jupyter-server-120c710>" ioloop oref))
  #f(compiled-function (object class slot-name fn) "Slot unbound is invoked during an attempt to reference an unbound slot.\nOBJECT is the instance of the object being reference.  CLASS is the\nclass of OBJECT, and SLOT-NAME is the offending slot.  This function\nthrows the signal `unbound-slot'.  You can overload this function and\nreturn the value to use in place of the unbound value.\nArgument FN is the function signaling this error.\nUse `slot-boundp' to determine if a slot is bound or not.\n\nIn CLOS, the argument list is (CLASS OBJECT SLOT-NAME), but\nEIEIO can only dispatch on the first argument, so the first two are swapped." #<bytecode 0x569ce5>)(#<jupyter-server jupyter-server-120c710> #s(eieio--class :name jupyter-server :docstring nil :parents (#s(eieio--class :name jupyter-rest-client :docstring nil :parents nil :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props ((:documentation . "The WebSocket url to use.\n\nIf this slot is not bou..."))) #s(cl-slot-descriptor :name auth :initform unbound :type t :props ((:documentation . "Indicator for authentication.\n\nIf the symbol ask, ...")))] :index-table #<hash-table eq 3/65 0x2f6e0ad> :children (jupyter-server) :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-rest-client jupyter-rest-client-938244> :options (:custom-groups nil)) #s(eieio--class :name jupyter-ioloop-comm :docstring nil :parents (#s(eieio--class :name jupyter-comm-layer :docstring nil :parents nil :slots [#s(cl-slot-descriptor :name clients :initform nil :type list :props nil)] :index-table #<hash-table eq 1/65 0x2eded19> :children (jupyter-server-kernel-comm jupyter-ioloop-comm) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-comm-layer jupyter-comm-layer-2edeff8> :options (:custom-groups nil :abstract t))) :slots [#s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil)] :index-table #<hash-table eq 2/65 0x22869c1> :children (jupyter-channel-ioloop-comm jupyter-server) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-ioloop-comm jupyter-ioloop-comm-2effed0> :options (:custom-groups nil :abstract t)) #s(eieio--class :name eieio-instance-tracker :docstring "This special class enables instance tracking.\nInhe..." :parents nil :slots [] :index-table #<hash-table eq 0/65 0x2e64cd1> :children (jupyter-server) :initarg-tuples nil :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform unbound :type symbol :props ((:documentation . "The symbol used to maintain a list of our instance...")))] :class-allocation-values [unbound] :default-object-cache #<eieio-instance-tracker eieio-instance-tracker-2e64fb8> :options (:custom-groups nil :documentation "This special class enables instance tracking.\nInhe..." :abstract t))) :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props ((:documentation . "The WebSocket url to use.\n\nIf this slot is not bou..."))) #s(cl-slot-descriptor :name auth :initform unbound :type t :props ((:documentation . "Indicator for authentication.\n\nIf the symbol ask, ..."))) #s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil) #s(cl-slot-descriptor :name kernelspecs :initform nil :type json-plist :props ((:documentation . "Kernelspecs for the kernels available behind this ...")))] :index-table #<hash-table eq 6/65 0xe606e5> :children nil :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform 'jupyter--servers :type symbol :props ((:documentation . "The symbol used to maintain a list of our instance...")))] :class-allocation-values [jupyter--servers] :default-object-cache #<jupyter-server jupyter-server-e60714> :options (:custom-groups nil)) ioloop oref)
  apply(#f(compiled-function (object class slot-name fn) "Slot unbound is invoked during an attempt to reference an unbound slot.\nOBJECT is the instance of the object being reference.  CLASS is the\nclass of OBJECT, and SLOT-NAME is the offending slot.  This function\nthrows the signal `unbound-slot'.  You can overload this function and\nreturn the value to use in place of the unbound value.\nArgument FN is the function signaling this error.\nUse `slot-boundp' to determine if a slot is bound or not.\n\nIn CLOS, the argument list is (CLASS OBJECT SLOT-NAME), but\nEIEIO can only dispatch on the first argument, so the first two are swapped." #<bytecode 0x569ce5>) #<jupyter-server jupyter-server-120c710> (#s(eieio--class :name jupyter-server :docstring nil :parents (#s(eieio--class :name jupyter-rest-client :docstring nil :parents nil :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props (...)) #s(cl-slot-descriptor :name auth :initform unbound :type t :props (...))] :index-table #<hash-table eq 3/65 0x2f6e0ad> :children (jupyter-server) :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-rest-client jupyter-rest-client-938244> :options (:custom-groups nil)) #s(eieio--class :name jupyter-ioloop-comm :docstring nil :parents (#s(eieio--class :name jupyter-comm-layer :docstring nil :parents nil :slots [...] :index-table #<hash-table eq 1/65 0x2eded19> :children (jupyter-server-kernel-comm jupyter-ioloop-comm) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-comm-layer jupyter-comm-layer-2edeff8> :options (:custom-groups nil :abstract t))) :slots [#s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil)] :index-table #<hash-table eq 2/65 0x22869c1> :children (jupyter-channel-ioloop-comm jupyter-server) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-ioloop-comm jupyter-ioloop-comm-2effed0> :options (:custom-groups nil :abstract t)) #s(eieio--class :name eieio-instance-tracker :docstring "This special class enables instance tracking.\nInhe..." :parents nil :slots [] :index-table #<hash-table eq 0/65 0x2e64cd1> :children (jupyter-server) :initarg-tuples nil :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform unbound :type symbol :props (...))] :class-allocation-values [unbound] :default-object-cache #<eieio-instance-tracker eieio-instance-tracker-2e64fb8> :options (:custom-groups nil :documentation "This special class enables instance tracking.\nInhe..." :abstract t))) :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props ((:documentation . "The WebSocket url to use.\n\nIf this slot is not bou..."))) #s(cl-slot-descriptor :name auth :initform unbound :type t :props ((:documentation . "Indicator for authentication.\n\nIf the symbol ask, ..."))) #s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil) #s(cl-slot-descriptor :name kernelspecs :initform nil :type json-plist :props ((:documentation . "Kernelspecs for the kernels available behind this ...")))] :index-table #<hash-table eq 6/65 0xe606e5> :children nil :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform 'jupyter--servers :type symbol :props ((:documentation . "The symbol used to maintain a list of our instance...")))] :class-allocation-values [jupyter--servers] :default-object-cache #<jupyter-server jupyter-server-e60714> :options (:custom-groups nil)) ioloop oref))
  slot-unbound(#<jupyter-server jupyter-server-120c710> #s(eieio--class :name jupyter-server :docstring nil :parents (#s(eieio--class :name jupyter-rest-client :docstring nil :parents nil :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props ((:documentation . "The WebSocket url to use.\n\nIf this slot is not bou..."))) #s(cl-slot-descriptor :name auth :initform unbound :type t :props ((:documentation . "Indicator for authentication.\n\nIf the symbol ask, ...")))] :index-table #<hash-table eq 3/65 0x2f6e0ad> :children (jupyter-server) :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-rest-client jupyter-rest-client-938244> :options (:custom-groups nil)) #s(eieio--class :name jupyter-ioloop-comm :docstring nil :parents (#s(eieio--class :name jupyter-comm-layer :docstring nil :parents nil :slots [#s(cl-slot-descriptor :name clients :initform nil :type list :props nil)] :index-table #<hash-table eq 1/65 0x2eded19> :children (jupyter-server-kernel-comm jupyter-ioloop-comm) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-comm-layer jupyter-comm-layer-2edeff8> :options (:custom-groups nil :abstract t))) :slots [#s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil)] :index-table #<hash-table eq 2/65 0x22869c1> :children (jupyter-channel-ioloop-comm jupyter-server) :initarg-tuples nil :class-slots [] :class-allocation-values [] :default-object-cache #<jupyter-ioloop-comm jupyter-ioloop-comm-2effed0> :options (:custom-groups nil :abstract t)) #s(eieio--class :name eieio-instance-tracker :docstring "This special class enables instance tracking.\nInhe..." :parents nil :slots [] :index-table #<hash-table eq 0/65 0x2e64cd1> :children (jupyter-server) :initarg-tuples nil :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform unbound :type symbol :props ((:documentation . "The symbol used to maintain a list of our instance...")))] :class-allocation-values [unbound] :default-object-cache #<eieio-instance-tracker eieio-instance-tracker-2e64fb8> :options (:custom-groups nil :documentation "This special class enables instance tracking.\nInhe..." :abstract t))) :slots [#s(cl-slot-descriptor :name url :initform "http://localhost:8888" :type string :props nil) #s(cl-slot-descriptor :name ws-url :initform unbound :type string :props ((:documentation . "The WebSocket url to use.\n\nIf this slot is not bou..."))) #s(cl-slot-descriptor :name auth :initform unbound :type t :props ((:documentation . "Indicator for authentication.\n\nIf the symbol ask, ..."))) #s(cl-slot-descriptor :name clients :initform nil :type list :props nil) #s(cl-slot-descriptor :name ioloop :initform unbound :type jupyter-ioloop :props nil) #s(cl-slot-descriptor :name kernelspecs :initform nil :type json-plist :props ((:documentation . "Kernelspecs for the kernels available behind this ...")))] :index-table #<hash-table eq 6/65 0xe606e5> :children nil :initarg-tuples ((:url . url) (:ws-url . ws-url) (:auth . auth)) :class-slots [#s(cl-slot-descriptor :name tracking-symbol :initform 'jupyter--servers :type symbol :props ((:documentation . "The symbol used to maintain a list of our instance...")))] :class-allocation-values [jupyter--servers] :default-object-cache #<jupyter-server jupyter-server-e60714> :options (:custom-groups nil)) ioloop oref)
  eieio-barf-if-slot-unbound(unbound #<jupyter-server jupyter-server-120c710> ioloop oref)
  slot-value(#<jupyter-server jupyter-server-120c710> ioloop)
  #f(compiled-function (comm) #<bytecode 0x2f51cf9>)(#<jupyter-server jupyter-server-120c710>)
  apply(#f(compiled-function (comm) #<bytecode 0x2f51cf9>) #<jupyter-server jupyter-server-120c710> nil)
  jupyter-comm-stop(#<jupyter-server jupyter-server-120c710>)
  jupyter-gc-servers()
  eval((jupyter-gc-servers) nil)
  eval-expression((jupyter-gc-servers) nil nil 127)
  funcall-interactively(eval-expression (jupyter-gc-servers) nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)

[nnicandro]

@dcherian Really appreciate all your feedback and bug finding! Its a big help in polishing up the notebook support.

jupyter-gc-servers seems good but isn't interactive? That could be the backup-nuke-everything-and-start-again option.

Yeah its not interactive right now because I am trying to have these kinds of automatic cleanup of resources without user intervention. There may be too many scenarios which are hard to recover from automatically to be able to do this though.

I get this traceback on eval-ing (jupyter-gc-servers)

Thanks I found the problem.

[dcherian]

@dcherian Really appreciate all your feedback and bug finding! Its a big help in polishing up the notebook support.

You're welcome. Thanks ofor all your work too. I'm a lot more productive in emacs than in the browser notebook interface!

jupyter-gc-servers seems good but isn't interactive? That could be the backup-nuke-everything-and-start-again option.

Yeah its not interactive right now because I am trying to have these kinds of automatic cleanup of resources without user intervention. There may be too many scenarios which are hard to recover from automatically to be able to do this though.

Yes, it's nice to have a worst-case option.

[nnicandro]

I've just pushed some commits to #127 that allow you to associate a name to a kernel so that you can distinguish kernels easily in the kernel list. See the relevant parts of the README and the jupyter-server-kernel-names variable for more details.

These features are also integrated with org-mode source blocks so if you specify :session as /jpy::thermo then thermo can either be:

• The name of an existing kernel found in jupyter-server-kernel-names, in which case the associated kernel ID will be used to connect the session to the kernel.
• A name that will be added to jupyter-server-kernel-names and will be associated with a newly launched kernel used for the session.

Feedback welcome.

[nnicandro]

I've gone ahead and merged #127. Feel free to open up any new issues pertaining to the notebook support.