Getting started with remote kernels

[hrishikeshac]

I recently moved from jupyter notebook-centric workflow to atom/ Hydrogen, and am absolutely loving it. I can run Hydrogen fine locally, but I haven't been able to successfully connect to remote kernels. I work with climate data, and the data and the scripts are typically located on a remote server. With jupyter notebook, I used to use ssh tunneling. I am not sure how to run those remote files remotely using local atom+hydrogen. Based on the documentation, I have tried a few things, but to no success.

1. How do I access remote files on local atom installation?

2. How do connect to a remote kernel? In this, I made some headway with ssh tunneling. a. On server: jupyter notebook --no-browser --port=7777 b. On local: ssh -N -f -L localhost:7777:localhost:7777 user@abc.xyz.edu c. Kernel Gateway setting: [{ "name": "server's fancy name", "options": {"baseUrl": "http://localhost:7777/"} }] d. Authenticated token when asked, and clicked on new session. But now I see a blank window with 'No Kernel Specs Available'

What do you think of what I've done so far? Any help/ suggestions/ comments/ please? I would really appreciate a very good, clear blog/ video going through the exact workflow to connect remotely.

[DoaneAS]

Hi,

I'm also having the same issue, and have followed much the same process as above.

@hrishikeshac one thing you can try is to verify that the ssh tunnel is working, by going to http://localhost:7777 in a browser. This works for me (using a port 9999), but I'm having the same error about no kernel specs available.

Can anyone suggest how to debug this? Seems like I may need to provide atom/hydrogen more information from the server kernelspec, but I'm not clear how to go about doing this.

[nikitakit]

Do you have the right type of file open when you try to connect to the remote kernel? Kernelspecs presented to you are filtered based on the grammar of the currently open file (so e.g. python files should only list python kernels)

You can also try opening a blank notebook from the web UI (which will create a session for it), and then see if the session is available to connect to from hydrogen.

Other options for debugging are opening the dev tools in atom, and also looking at the server logs (which the notebook process prints to stdout) if those are available to you.

[DoaneAS]

Thanks @nikitakit, having a python script open in atom, as well as using jupyter kernelgateway package on my remote server, fixed the problem and I was able to connect to a python2 or python3 kernel on the remote server.

Before using jupyter kernelgateway, I was using jupyter notebook or jupyter lab, with the port and token defined in the respective config file on the remote server. I also reinstalled atom.

Thanks for the helpful suggestion.

[jkroes]

@DoaneAS Your use of kernel_gateway was the only documented way I have found to use Hydrogen with a remote kernel.

[KirkHadley]

Hey folks just wanted to document that I just got this error trying to connect to a remote jupyter notebook server NotebookApp.token = '<generated' (i.e. default, require a token) and got things to work simply by opening a .py file in atom and attempting the remote kernel connect process a second time. Perhaps no need to change over to jupyer kernelgateway and/or run an insecure notebook server?

[max-sixty]

I'm having issues with jupyter-notebook when using both a) http and b) tokens. I can't authenticate by typing the token into the login in the browser, let alone connect through hydrogen.

I've followed all the steps in the 'Remote Kernel Connection' docs (which I wrote most of). Potentially there's an upstream change that doesn't allow http & tokens to work together?

[zachlipp]

Is there any movement on this? I've been a big fan of hydrogen but haven't been able to use it for months

[zzz123xyz]

run the jupyter notebook with the --ip=full ip address of your server. Then just fill the full ip address of your server:port and write all the specs in a single line in the kernel gateways blank. Then it works

[brucemoran]

Just gone through setup and have it working on jupyter notebook

Wanted to leave this to remind in case of required re-setup.

I am running in a Singularity container on a remote cluster, working on node called 'node01'.

Server:

1. edit ~/.jupyter/jupyter_notebook_config.py, commenting in c.NotebookApp.token = 'my_new_token_that_I_wrote'
2. run jupyter notebook, here for my install singularity exec -B /path/to/my/jupyter_notebook:/home/bmoran/jupyter_notebook /path/to/singularity/containter.for.analysis.sif jupyter-notebook --no-browser --port=8891 --ip=127.0.0.1

Local:

1. ssh tunnel to server ssh -N -f -L 127.0.0.1:8891:127.0.0.1:8891 node01 -v -v

Should be able to copy/paste the URL printed to screen on server and connect locally in browser now.

Atom:

0. Packages -> Settings -> Manage Packages -> Hydrogen Settings [[scroll to Kernel Gateways, adding following]] [{ "name": "node01", "options": { "baseUrl": "http://127.0.0.1:8891", "token": "my_new_token_that_I_wrote" } }] and restart Atom
1. Open a file in Atom that you want to run in the kernel that you are connected to in browser
2. Packages -> Hydrogen -> Connect to Remote Kernel [[select a gateway, click node01, enter token]]
3. The filename open in the browser should be available to select, and run in kernel, edit and save in your current script. NB that any script in Atom can now be run in kernel...very nice!