Auto-start remote kernel not working

[dangom]

Currently my workflow has consisted of calling python -m ipykernel_launcher -f ~/remote.json on a remote computer, and then connecting to ia via an Org block specifying :session /ssh:remote:~/remote.json.

I've tried a couple of times emacs-jupyter auto-spawn feature, but never really got it to work. I think it's probably an issue at my end, but if someone has a hint, I'd appreciate it.

When I do :session /ssh:remote:test, I get an error saying (error "Can’t obtain kernelspecs from jupyter shell command"). My tramp config has (add-to-list 'tramp-remote-path 'tramp-own-remote-path), but that doesn't seem to make a difference. Is there something I have to do on the remote side in order for jupyter to find the jupyter command? In my .bashrc or .zshrc I initialize conda via the conda provided initialize setup.

[nnicandro]

What does the following return?

(let ((default-directory "/ssh:remote:"))
  (with-temp-buffer
    (process-file "jupyter" nil t nil "kernelspec" "list")
    (buffer-string)))

This is probably another case where better logging would reveal the problem.

[dangom]

Yes, seems like jupyter is not found. =/ I don't have admin rights on these remote machines, so I can't symlink jupyter in /usr/local, unfortunately.

#+begin_src emacs-lisp
(let ((default-directory "/ssh:remote:"))
  (with-temp-buffer
    (process-file "jupyter" nil t nil "kernelspec" "list")
    (buffer-string)))
#+end_src

#+RESULTS:
: sh: jupyter: command not found

Is there a better way to have jupyter-emacs find the remote emacs without sourcing any files. Maybe a remote .config/jupyter-emacs/config.txt with the output of jupyter kernelspec list?

[nnicandro]

Maybe add the conda stuff in ~/.profile on the remote. It seems TRAMP is using sh on the remote so the bash specific setup in ~/.bashrc is most likely not being loaded.

Alternatively you could probably modify the tramp-remote-shell variable in the ssh method of tramp-methods to use bash.

[dangom]

Solved. Had to look into this SO answer about setting PATHs for sh & tramp to get it to work correctly.

Thanks!!

[dangom]

There is still an edge case where it doesn't seem to work if I double hop via a ProxyJump (in ssh config). This works :session /ssh:remote:test, this too/ssh:entry|ssh:remote:test, but this doesn't: /ssh:remote-tunnel:test. The kernel doesn't seem to respond to kernel-info request.

# ssh config
Host remote-tunnel
     User user
     ProxyJump entry
     HostName remote

[NightMachinery]

@nnicandro When I try to start remote kernels, I get this error:

executing Jupyter-Julia code block...
Kernel process exited:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'jupyter_client'

finalizer failed: (cl-no-primary-method jupyter-kill-kernel #s(jupyter-kernel-process-manager (#<finalizer used>) jupyter--kernel-managers #s(jupyter-command-kernel (#<finalizer used>) ("julia-1.6" "/ssh:ubuntu@194.5.193.126:/home/ubuntu/.local/share/jupyter/kernels/julia-1.6" :argv ["/home/ubuntu/packages/julias/julia-1.6/bin/julia" "-i" "--color=yes" "--project=@." "/home/ubuntu/.julia/packages/IJulia/e8kqU/src/kernel.jl" "{connection_file}"] :env nil :display_name "Julia 1.6.3" :language "julia" :interrupt_mode "signal" :metadata nil) eieio--unbound #<process jupyter-kernel-julia-1.6>) nil))

But this works:

#+begin_src elisp :results value :exports both :wrap example
(let ((default-directory "/ssh:ubuntu@194.5.193.126:"))
  (with-temp-buffer
    (process-file "timeout" nil t nil "10s" "jupyter" "kernel" "--kernel=julia-1.6")
    (buffer-string)))
#+end_src

#+RESULTS:
#+begin_example
[KernelApp] Starting kernel 'julia-1.6'
[KernelApp] Connection file: /home/ubuntu/.local/share/jupyter/runtime/kernel-02fef061-b50f-48b2-b251-02c3d7a0711c.json
[KernelApp] To connect a client: --existing kernel-02fef061-b50f-48b2-b251-02c3d7a0711c.json
[KernelApp] Shutting down on signal 15
#+end_example

#+begin_src elisp :results value :exports both :wrap example
(let ((default-directory "/ssh:ubuntu@194.5.193.126:"))
  (with-temp-buffer
    (process-file "timeout" nil t nil "15s" "/home/ubuntu/packages/julias/julia-1.6/bin/julia" "-i" "--color=yes" "--project=@." "/home/ubuntu/.julia/packages/IJulia/e8kqU/src/kernel.jl")
    (buffer-string)))
#+end_src

#+RESULTS:
#+begin_example
connect ipython with --existing /home/ubuntu/profile-58847.json
Starting kernel event loops.
#+end_example

What diagnostics can I try?

---

I have also tried

(add-to-list 'tramp-remote-process-environment "PYTHONPATH='/home/ubuntu/miniconda3/lib/python3.9/site-packages'")

but either tramp is not using this variable (it doesn't seem to use tramp-default-remote-shell), or setting PYTHONPATH is useless.