search

emacs-lsp / lsp-pyright

lsp-mode :heart: pyright
https://emacs-lsp.github.io/lsp-pyright
GNU General Public License v3.0
284 stars 23 forks source link

Document/streamline pyright/lsp-mode over TRAMP #10

Closed mjlbach closed 4 years ago

mjlbach commented 4 years ago

I've seen several comments on the github issues asking about tramp, python virtual environments and lsp-mode for python. It's currently possible-ish with a couple hacks. For pyright (for example), the following launches uses the standard python interpreter /usr/bin/python

   (setq lsp-log-io t)
   (setq lsp-pyright-use-library-code-for-types t)
   (setq lsp-pyright-diagnostic-mode "workspace")
   (lsp-register-client
     (make-lsp-client
       :new-connection (lsp-tramp-connection (lambda ()
                                       (cons "pyright-langserver"
                                             lsp-pyright-langserver-command-args)))
       :major-modes '(python-mode)
       :remote? t
       :server-id 'pyright-remote
       :multi-root t
       :priority 3
       :initialization-options (lambda () (ht-merge (lsp-configuration-section "pyright")
                                                    (lsp-configuration-section "python")))
       :initialized-fn (lambda (workspace)
                         (with-lsp-workspace workspace
                           (lsp--set-configuration
                           (ht-merge (lsp-configuration-section "pyright")
                                     (lsp-configuration-section "python")))))
       :download-server-fn (lambda (_client callback error-callback _update?)
                             (lsp-package-ensure 'pyright callback error-callback))
       :notification-handlers (lsp-ht ("pyright/beginProgress" 'lsp-pyright--begin-progress-callback)
                                     ("pyright/reportProgress" 'lsp-pyright--report-progress-callback)
                                     ("pyright/endProgress" 'lsp-pyright--end-progress-callback))))

To accomodate virtualenvs, I added some additional initialization code to read the virtualenvironment off of pyvenv, prune the tramp path, and pass it to the server.

    :initialization-options (lambda () (let* ((pyright_hash (lsp-configuration-section "pyright"))
                                               (python_hash (lsp-configuration-section "python"))
                                               (_ (puthash "pythonPath" (concat (replace-regexp-in-string (file-remote-p default-directory) "" pyvenv-virtual-env) "bin/python") (gethash "python" python_hash))))
                                          (ht-merge pyright_hash
                                                    python_hash)))
    :initialized-fn (lambda (workspace)
                      (with-lsp-workspace workspace
                        (lsp--set-configuration
                        (let* ((pyright_hash (lsp-configuration-section "pyright"))
                                               (python_hash (lsp-configuration-section "python"))
                                               (_ (puthash "pythonPath" (concat (replace-regexp-in-string (file-remote-p default-directory) "" pyvenv-virtual-env) "bin/python") (gethash "python" python_hash))))
                                          (ht-merge pyright_hash
                                                    python_hash)))))

This could be cleaner, and there isn't a unified way with pyenv-mode and poetry.el not officially supporting virtualenvs over TRAMP. I have a version of virtualenv support for pyvenv/poetry.el, but maybe it would make sense to coordinate with them upstream.

seagle0128 commented 4 years ago

I am not using the remote env to develop and test Python files. Would you please create a PR?

mjlbach commented 4 years ago

I can make something for the documentation, I'll work on trying to upstream something with pyenv, poetry.el, and pipenv.

seagle0128 commented 4 years ago

@mjlbach can u try the latest version?

mjlbach commented 4 years ago

I don't think anything pertinent has changed? I'm on the latest version. The above snippet I provided still works given you set the appropriate remote interpreter with pyvenv. This is more a documentation issue (I have my qualifying exams this week so I haven't been able to work on this).

The bigger ecosystem issue pyvenv/poetry don't support remote virtual environments. What I did above was a hack where I just use pyvenv and filter out the tramp prepended path to pass it to the server initialization options.

mpereira commented 4 years ago

@mjlbach thanks for sharing your setup, it works for me as well!

The following clients were selected based on priority: (server-id pyright-remote, priority 3)
Pyright language server 1.1.63 starting
Server root directory: /usr/local/lib/node_modules/pyright/dist/
No configuration file found.
Setting pythonPath for service "<REDACTED>": "/usr/bin/python"
stubPath <REDACTED>/typings is not a valid directory.
Python version 2.7 from interpreter is unsupported
Assuming Python platform Linux
Searching for source files
Found 22 source files
tshu-w commented 4 years ago

@mjlbach Thank you so much for saving my time. For those who use conda, can refer to https://github.com/emacs-lsp/lsp-pyright/issues/7 to set up python.pythonPath and python.venvPath in emacs or pyrightconfig.json

tshu-w commented 4 years ago

@seagle0128 Here's a small problem. python.pythonPath was not set correctly when I edit tramp buffers. since lsp-pyright-locate-python cannot get remote python's path. I tried to comment on this line L197 of code, pyright works both on local or remote. Is there a point to this line of code. Is it possible to remove or make lsp-pyright-locate-python support finding remote python by simply adding t as an argument to executable-find(e.g. (executable-find lsp-pyright-python-executable-cmd t))

seagle0128 commented 4 years ago

Thanks, @tshu-w . I updated lsp-pyright-locate-python to search remote python cmd.

yyoncho commented 4 years ago

Just want to mention that the reason for asking users to create their remote registration is the lack of manpower to support one more configuration mode. So, in this case, if there is interest IMO the tramp configuration might live in the repo.

mjlbach commented 4 years ago

@seagle0128 I think this will only work if tramp's remote path is amended to prepend the virtual environment path. It doesn't, for example, read from the virtual environment path set by pyenv like the snippet I posted does.

seagle0128 commented 4 years ago

@seagle0128 I think this will only work if tramp's remote path is amended to prepend the virtual environment path. It doesn't, for example, read from the virtual environment path set by pyenv like the snippet I posted does.

The commit doesn't resolve the remote venv issue. As you mentioned, it needs upstreams supports. Anyway, the PRs are welcome.

mjlbach commented 4 years ago

Just to concretize what needs to be upstreamed (for the benefit of others):

I have a dirty hack that works with poetry.el to support this:

diff --git a/poetry.el b/poetry.el
index 275ce85..2188120 100644
--- a/poetry.el
+++ b/poetry.el
@@ -681,7 +681,7 @@ compilation buffer name."
                                default-directory)))
     (unless (member command '(new init config))
       (poetry-ensure-in-project))
-    (let* ((prog (or (executable-find "poetry")
+    (let* ((prog (or (executable-find "poetry" t)
                      (poetry-error "Could not find 'poetry' executable")))
            (args (if (or (string= command "run")
                          (string= command "config")
@@ -896,7 +896,9 @@ If OPT is non-nil, set an optional dep."
                        ".venv")
              ;; virtualenvs elsewhere
              (car (directory-files
-                   (poetry-get-configuration "virtualenvs.path")
+                   (concat (file-remote-p default-directory)
+                    (poetry-get-configuration "virtualenvs.path")
+                   )
                    t
                    (format "^%s-.+-py.*$"
                            (downcase (replace-regexp-in-string "_" "-" (poetry-get-project-name)))))))
jdtsmith commented 3 years ago

I've had mixed success with lsp-pyright over tramp. I sorted out some path issues (system node is ancient and was being used by pyright), but most of the time pyright starts but then does not appear to communicate:

Command "pyright-langserver --stdio" is present on the path.
Command "pyright-langserver --stdio" is present on the path.
Found the following clients for /ssh:XXX:/YYY/ZZZ.py: (server-id pyright-remote, priority 3)
The following clients were selected based on priority: (server-id pyright-remote, priority 3)
Pyright language server 1.1.111 starting
Server root directory: /usr/local/lib/node_modules/pyright/dist/

But then... nothing, and nothing on stderr in /tmp/pyright-remote-1-stderr, and no capabilities in lsp-describe-session. When it has worked, it occasionally locks up hard, and a pkill -SIGUSR2 Emacs is needed to recover control. In this case it's usually hanging with tramp waiting for process output. While pyright-langserver is stuck doing nothing, other tramp commands still work fine.

Has anyone had this experience or have suggestions to overcome it?

tshu-w commented 3 years ago

@jdtsmith There are some discussions at https://github.com/emacs-lsp/lsp-mode/issues/2375 https://github.com/emacs-lsp/lsp-mode/issues/1845

jdtsmith commented 3 years ago

Thanks. I see those are related to \n conversion, but here I don't even see any output to stderr or otherwise, so it's hard to conclude it's the same thing. The pyright-langserver is running but not communicating. It would be useful to have a simple "ping the server" command in emacs-lsp to see if the connection is actually functioning and producing sensible results. What's strange is that rarely, sometimes after an lsp-workspace-restart, sometimes on a fresh start, the server seems to function for a while, usually locking up after that in an apparent tramp-related blockage.

tshu-w commented 3 years ago

@jdtsmith In fact I was troubled by the same problem as you before, many times no error was reported, just stuck and a reboot might work. Since upgrading to emacs28 this phenomenon has been significantly alleviated.

jdtsmith commented 3 years ago

Interesting. Do you believe this is due to Emacs 28 improvements with tramp? I have set tramp-verbose to 7 or so to watch the traffic, but there's nothing obviously broken there. Either information is flowing to the server, or it is not.

I have wondered whether the watch file capability is freezing pyright. When it works it always says:

Creating watch for /ssh:XXX:/XXX/YYY/ZZZ.py
Failed to create a watch for No file notification program found on /ssh:XXX:: message

Does anyone have a suggestion to ping/probe a running server separate from Emacs/tramp, to check whether it's a LSP server issue or an Emacs/tramp issue? Equivalent to telnetting to port 80 to see if an httpd server is running & communicating...

tshu-w commented 3 years ago

Do you believe this is due to Emacs 28 improvements with tramp

Yes, according to here https://github.com/emacs-lsp/lsp-mode/issues/2375#issuecomment-749132863:

Michael fixed the issue in the tramp code

You can run it directly on the server, I'm not sure how to communicate with it, but I suggest discussing it directly at lsp-mode, I don't think it's a pyright issue.

jdtsmith commented 3 years ago

I found that lsp-mode PR #2531 did the trick for me (+ upgrading Tramp to v2.5). I didn't need to update Emacs itself.

xukai92 commented 3 years ago

I found that lsp-mode PR #2531 did the trick for me (+ upgrading Tramp to v2.5). I didn't need to update Emacs itself.

Just want to confirm that this works to me as well.

jingxuanlim commented 3 years ago

Hello

I've followed instructions on this thread and, as @tshu-w suggested, here, to setup lsp-mode with conda and pyright on a remote server, but I am still running into problems.

LSP :: There are no language servers supporting current mode `python-mode' registered with `lsp-mode'.
This issue might be caused by:
1. The language you are trying to use does not have built-in support in `lsp-mode'. You must install the required support manually. Examples of this are `lsp-java' or `lsp-metals'.
2. The language server that you expect to run is not configured to run for major mode `python-mode'. You may check that by checking the `:major-modes' that are passed to `lsp-register-client'.
3. `lsp-mode' doesn't have any integration for the language behind `python-mode'. Refer to https://emacs-lsp.github.io/lsp-mode/page/languages and https://langserver.org/ .
4. You are over `tramp'. In this case follow https://emacs-lsp.github.io/lsp-mode/page/remote/.
5. You have disabled the `lsp-mode' clients for that file. (Check `lsp-enabled-clients' and `lsp-disabled-clients').

This is in my doom-emacs config.el, which is basically what @mjlbach suggested.

(setq enable-remote-dir-locals t)
(setq enable-local-variables :all)

(use-package! lsp-mode
  :commands lsp
  :hook
  (python-mode . lsp))

(after! lsp-clients
  (setq lsp-log-io t)
  (setq lsp-pyright-use-library-code-for-types t)
  (setq lsp-pyright-diagnostic-mode "workspace")
  (lsp-register-client
    (make-lsp-client
      :new-connection (lsp-tramp-connection (lambda ()
                                      (cons "pyright-langserver"
                                            lsp-pyright-langserver-command-args)))
      :major-modes '(python-mode)
      :remote? t
      :server-id 'pyright-remote
      :multi-root t
      :priority 3
      :initialization-options (lambda () (ht-merge (lsp-configuration-section "pyright")
                                                   (lsp-configuration-section "python")))
      :initialized-fn (lambda (workspace)
                        (with-lsp-workspace workspace
                          (lsp--set-configuration
                          (ht-merge (lsp-configuration-section "pyright")
                                    (lsp-configuration-section "python")))))
      :download-server-fn (lambda (_client callback error-callback _update?)
                            (lsp-package-ensure 'pyright callback error-callback))
      :notification-handlers (lsp-ht ("pyright/beginProgress" 'lsp-pyright--begin-progress-callback)
                                    ("pyright/reportProgress" 'lsp-pyright--report-progress-callback)
                                    ("pyright/endProgress" 'lsp-pyright--end-progress-callback))))
  )

In my project directory, I made a new .dir-locals.el file with the follow contents, like what @tshu-w suggested in the other thread.

((python-mode . ((eval . (lsp-register-custom-settings
                          '(("python.pythonPath" "~/anaconda3/envs/python37/bin/python"
                             "python.venvPath" "~/anaconda3/envs/python37")))))))

Anyone knows what I'm missing?

@mjlbach, I note that you have tried this with doom-emacs: https://github.com/hlissner/doom-emacs/issues/3390#issuecomment-667708051. Anything I should take note of?

tshu-w commented 3 years ago

@jdtsmith You should paste the content of *lsp-log*. Does every thing work fine on local. What's the results if you run M-: (executable-find "pyright" t) when open tramp buffers. My current guess is that you may need the following config:

(add-to-list 'tramp-remote-path 'tramp-own-remote-path)
jingxuanlim commented 3 years ago

Hi @tshu-w. Thanks for responding!

I don't have lsp setup locally as I do all my work on a remote filesystem so I don't even have my conda env etc on my local machine.

I couldn't find *lsp-log* in my list of buffers after opening a remote .py file.

Running M-: (executable-find "pyright" t) returns nil.

Interestingly, I couldn't add that line to my doom-emacs config.el. Here's the backtrace.

  (member 'tramp-own-remote-path tramp-remote-path)
  (if (member 'tramp-own-remote-path tramp-remote-path) tramp-remote-path (setq tramp-remote-path (cons 'tramp-own-remote-path tramp-remote-path)))
  eval-buffer(#<buffer  *load*-149563> nil "/home/user/.doom.d/config.el" nil t)  ; Reading at buffer position 3775
  load-with-code-conversion("/home/user/.doom.d/config.el" "/home/user/.doom.d/config.el" t t)
  load("/home/user/.doom.d/config" t nomessage)
  (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage))
  (condition-case e (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name "config" doom-private-dir) doom-private-dir)))
  (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name "config" doom-private-dir) doom-private-dir))) (if custom-file (progn (load custom-file 'noerror (not doom-debug-mode)))))
  (progn (if doom-debug-p (progn (let ((inhibit-message (active-minibuffer-window))) (message #("DOOM Initializing user config" 0 5 (face font-lock-comment-face)))))) (maphash (doom-module-loader doom-module-init-file) doom-modules) (doom-run-hooks 'doom-before-init-modules-hook) (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name "config" doom-private-dir) doom-private-dir))) (if custom-file (progn (load custom-file 'noerror (not doom-debug-mode))))))
  (if init-p (progn (if doom-debug-p (progn (let ((inhibit-message (active-minibuffer-window))) (message #("DOOM Initializing user config" 0 5 (face font-lock-comment-face)))))) (maphash (doom-module-loader doom-module-init-file) doom-modules) (doom-run-hooks 'doom-before-init-modules-hook) (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name "config" doom-private-dir) doom-private-dir))) (if custom-file (progn (load custom-file 'noerror (not doom-debug-mode)))))) nil)
  (let* ((init-p (and t (condition-case e (let (file-name-handler-alist) (load (expand-file-name doom-module-init-file doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name doom-module-init-file doom-private-dir) doom-private-dir)))))) (if init-p (progn (if doom-debug-p (progn (let ((inhibit-message ...)) (message #("DOOM Initializing user config" 0 5 ...))))) (maphash (doom-module-loader doom-module-init-file) doom-modules) (doom-run-hooks 'doom-before-init-modules-hook) (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let (file-name-handler-alist) (load (expand-file-name "config" doom-private-dir) t 'nomessage)) ((debug doom-error) (signal (car e) (cdr e))) ((debug error) (doom--handle-load-error e (expand-file-name "config" doom-private-dir) doom-private-dir))) (if custom-file (progn (load custom-file 'noerror (not doom-debug-mode)))))) nil))
  (progn (setq doom-init-modules-p t) (if no-config-p nil (if doom-debug-p (progn (let ((inhibit-message (active-minibuffer-window))) (message #("DOOM Initializing core modules" 0 5 (face font-lock-comment-face)))))) (doom-initialize-core-modules)) (let* ((init-p (and t (condition-case e (let (file-name-handler-alist) (load ... t ...)) ((debug doom-error) (signal ... ...)) ((debug error) (doom--handle-load-error e ... doom-private-dir)))))) (if init-p (progn (if doom-debug-p (progn (let (...) (message ...)))) (maphash (doom-module-loader doom-module-init-file) doom-modules) (doom-run-hooks 'doom-before-init-modules-hook) (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let (file-name-handler-alist) (load ... t ...)) ((debug doom-error) (signal ... ...)) ((debug error) (doom--handle-load-error e ... doom-private-dir))) (if custom-file (progn (load custom-file ... ...))))) nil)))
  (if (or force-p (not doom-init-modules-p)) (progn (setq doom-init-modules-p t) (if no-config-p nil (if doom-debug-p (progn (let ((inhibit-message ...)) (message #("DOOM Initializing core modules" 0 5 ...))))) (doom-initialize-core-modules)) (let* ((init-p (and t (condition-case e (let ... ...) (... ...) (... ...))))) (if init-p (progn (if doom-debug-p (progn (let ... ...))) (maphash (doom-module-loader doom-module-init-file) doom-modules) (doom-run-hooks 'doom-before-init-modules-hook) (if no-config-p nil (maphash (doom-module-loader doom-module-config-file) doom-modules) (doom-run-hooks 'doom-init-modules-hook) (condition-case e (let ... ...) (... ...) (... ...)) (if custom-file (progn ...)))) nil))))
  doom-initialize-modules()
  eval-buffer(#<buffer  *load*-168617> nil "/home/user/doom-emacs/init.el" nil t)  ; Reading at buffer position 2804
  load-with-code-conversion("/home/user/doom-emacs/init.el" "/home/user/doom-emacs/init.el" t t)
  load("/home/user/doom-emacs/init.el" t t)
  (let ((init-file (expand-file-name "init.el" user-emacs-directory))) (load init-file t t) (let ((chemacs-custom-file (chemacs-profile-get 'custom-file init-file))) (if (not custom-file) (progn (setq custom-file chemacs-custom-file) (if (equal custom-file init-file) nil (if (file-exists-p custom-file) nil (let (...) (save-current-buffer ... ...))) (load custom-file))))))
  chemacs-load-user-init()
  eval-buffer(#<buffer  *load*> nil "/home/user/.emacs.d/init.el" nil t)  ; Reading at buffer position 238
  load-with-code-conversion("/home/user/.emacs.d/init.el" "/home/user/.emacs.d/init.el" t t)
  load("/home/user/.emacs.d/init" noerror nomessage)
  startup--load-user-init-file(#f(compiled-function () #<bytecode 0x156ad29bc331>) #f(compiled-function () #<bytecode 0x156ad29bc349>) t)
  command-line()
  normal-top-level()

Any idea what's wrong?

jingxuanlim commented 3 years ago

@tshu-w, okay I fixed it with:

(after! tramp
  (add-to-list 'tramp-remote-path 'tramp-own-remote-path))

However, running M-: (executable-find "pyright" t) still returns nil.

tshu-w commented 3 years ago

The current problem is that Emacs cannot find your remote pyright. Can you run pyright on remote shell?

In my project directory, I made a new .dir-locals.el file with the follow contents, like what @tshu-w suggested in the other thread.

BTW, .dir-locals.el is only needed when you need a non-base environment if you can find the python path via executable-find.

jingxuanlim commented 3 years ago

Yes, I don't use the base environment on conda, so I guess .dir-locals.el is required.

Here I attempt to run pyright on the remote shell (but not from within a shell in emacs) after I activate the appropriate conda environment python37. Looks like there are also problems with pyright itself? It's my first time using, so I'm not sure.

$ ssh ws1
$ conda activate python37
(python37) $ pyright
No configuration file found.
No pyproject.toml file found.
stubPath ~/typings is not a valid directory.
Assuming Python platform Linux
Searching for source files
Skipping broken link "~/.cache/edm/packages/1c/09b80682943d242f6eee3a79276a18c76d7962c7e49e329e98179f3762abce/traitsui-5.1.0-2.egg/traitsui/image/library/icons.zip"
Skipping broken link "~/.cache/edm/packages/1c/09b80682943d242f6eee3a79276a18c76d7962c7e49e329e98179f3762abce/traitsui-5.1.0-2.egg/traitsui/image/library/std.zip"
Skipping broken link "~/.cache/edm/packages/8b/8df08a725117b8d62702fa63888f9ea96645607d889c5e402dbb4d51049b9c/statsmodels-0.8.0-2.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,csv,car,Duncan.csv.zip"
Skipping broken link "~/.cache/edm/packages/8b/8df08a725117b8d62702fa63888f9ea96645607d889c5e402dbb4d51049b9c/statsmodels-0.8.0-2.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,datasets.csv.zip"
Skipping broken link "~/.cache/edm/packages/8b/8df08a725117b8d62702fa63888f9ea96645607d889c5e402dbb4d51049b9c/statsmodels-0.8.0-2.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,doc,car,rst,Duncan.rst.zip"
Skipping broken link "~/.cache/edm/packages/97/b41e8af02f727c02e989d8c7aaa8b3dd5ace895dbed551255c4c139fc468c1/enable-4.6.0-8.egg/enable/images.zip"
Skipping broken link "~/.cache/edm/packages/b7/f470df7779d5cdd243ec80d24dd30b0398e453d167d010b1d225a5a651a7c8/pyproj-1.9.4-2.egg/pyproj/data/proj-datumgrid-1.5.zip"
Skipping broken link "~/.cache/edm/packages/d3/5442b5eb8a3ea8720369f6693cfec1339098333047fd1390ae6c32b8b733dc/mayavi-4.5.0-4.egg/tvtk/tvtk_classes.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/03/468968d57d86ad6b713a23c111111ee9f729a1f3f27b28030131238fd07e8b/enable-4.7.1-2.egg/enable/images.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/0c/973bfbc153114e8009f1112f0b3386efcf60aec9866ad01c7002bf59c82ad6/pandas-0.23.1-1.egg/pandas/tests/io/json/data/tsframe_v012.json.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/0c/973bfbc153114e8009f1112f0b3386efcf60aec9866ad01c7002bf59c82ad6/pandas-0.23.1-1.egg/pandas/tests/io/parser/data/salaries.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/0c/973bfbc153114e8009f1112f0b3386efcf60aec9866ad01c7002bf59c82ad6/pandas-0.23.1-1.egg/pandas/tests/io/parser/data/utf16_ex_small.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/34/34464d58efc74c8a2defe5ab405d61100a902d9b933e8ce9e177781cb80ca8/enable-4.7.2-1.egg/enable/images.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/64/024e89c78085d21d70a03dc9e44b4c2eb248136bfc9bad1293beb4f3668f80/statsmodels-0.8.0-5.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,csv,car,Duncan.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/64/024e89c78085d21d70a03dc9e44b4c2eb248136bfc9bad1293beb4f3668f80/statsmodels-0.8.0-5.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,datasets.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/64/024e89c78085d21d70a03dc9e44b4c2eb248136bfc9bad1293beb4f3668f80/statsmodels-0.8.0-5.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,doc,car,rst,Duncan.rst.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/69/21fba32132ba63da3f23e602864be3410c02bbb02675a56a7238d32eb094bd/traitsui-6.0.0-1.egg/traitsui/image/library/icons.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/69/21fba32132ba63da3f23e602864be3410c02bbb02675a56a7238d32eb094bd/traitsui-6.0.0-1.egg/traitsui/image/library/std.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/6a/d09d96a2c1782468884a03837e4a0d59a083f34037f3332106db77383ef968/statsmodels-0.8.0-6.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,csv,car,Duncan.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/6a/d09d96a2c1782468884a03837e4a0d59a083f34037f3332106db77383ef968/statsmodels-0.8.0-6.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,datasets.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/6a/d09d96a2c1782468884a03837e4a0d59a083f34037f3332106db77383ef968/statsmodels-0.8.0-6.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,doc,car,rst,Duncan.rst.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/75/78d80e79577f0aec332145c17df540921cdc324fc33f7eb7ad352c9d6dbea7/statsmodels-0.8.0-3.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,csv,car,Duncan.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/75/78d80e79577f0aec332145c17df540921cdc324fc33f7eb7ad352c9d6dbea7/statsmodels-0.8.0-3.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,datasets.csv.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/75/78d80e79577f0aec332145c17df540921cdc324fc33f7eb7ad352c9d6dbea7/statsmodels-0.8.0-3.egg/statsmodels/datasets/tests/raw.github.com,vincentarelbundock,Rdatasets,master,doc,car,rst,Duncan.rst.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/be/6727fc4fc031a60849343244c3e51586ce5c2a967af81c218b89dc048d5f9d/mayavi-4.5.0-5.egg/tvtk/tvtk_classes.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/c1/4f6191b3cfff0f026ff08e6e084ca300f98d2fba224f4dbe4c6c79859aa4d3/pyface-6.0.0-1.egg/pyface/image/library/icons.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/c1/4f6191b3cfff0f026ff08e6e084ca300f98d2fba224f4dbe4c6c79859aa4d3/pyface-6.0.0-1.egg/pyface/image/library/std.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/cf/54faadbda0fb2eec1a591a33f92991235559f7c39f551a1dc492b3835cb518/mayavi-4.5.0-6.egg/tvtk/tvtk_classes.zip"
Skipping broken link "~/.local/share/canopy/edm/canopy-platform-cache/packages/f3/c15dcfc01e65e1e74eef7a25a3e5b970b34b727dc44dfcc095ee5e3bc3e890/enable-4.7.2-2.egg/enable/images.zip"
Auto-excluding ~/.local/share/canopy/edm/envs/User
Auto-excluding ~/.local/share/canopy/edm/envs/mika_prepro
Auto-excluding ~/.local/share/canopy/edm/envs/python_3-5
(node:94823) UnhandledPromiseRejectionWarning: Libzip Error: Not a zip archive
    at Q.makeLibzipError (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ZipFS.js:149:29)
    at new Q (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ZipFS.js:121:28)
    at getZipSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ZipOpenFS.js:784:28)
    at g.makeCallSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ZipOpenFS.js:665:21)
    at g.existsSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ZipOpenFS.js:232:21)
    at a.existsSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ProxiedFS.js:71:28)
    at o.existsSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/node_modules/@yarnpkg/fslib/lib/ProxiedFS.js:71:28)
    at T.existsSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/src/common/fileSystem.ts:213:23)
    at t.PyrightFileSystem.existsSync (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/src/pyrightFileSystem.ts:63:29)
    at _configOptions.autoExcludeVenv.i.some.t (~/anaconda3/envs/python37/lib/node_modules/pyright/dist/pyright-internal/src/analyzer/service.ts:1027:53)
(node:94823) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:94823) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
$
tshu-w commented 3 years ago

@jingxlim Does you install pyright in conda env? I think you should try to add its location to PATH. Or install globally should be a more simpler choice, as (executable-find "pyright" t) cannot find pyright in your env.

jingxuanlim commented 3 years ago

Yes, I did install pyright in my conda env. Perhaps I mistook how pyright works, but I assumed that pyright needs to be aware of the packages installed in a certain environment and therefore needs to be installed in an environment where all I packages I use/need are installed.

In any case, I added my conda env's bin dir to $PATH and now emacs can find pyright!

"~/anaconda3/envs/python37/bin/pyright"

I'm still encountering problems with lsp-mode though. Same error as before.

By the way, should .dir-locals.el contain a tramp command? i.e. with /ssh:[remote]:

((python-mode . ((eval . (lsp-register-custom-settings
                          '(("python.pythonPath" "/ssh:[remote]:~/anaconda3/envs/python37/bin/python"
                             "python.venvPath" "/ssh:[remote]:~/anaconda3/envs/python37")))))))
tshu-w commented 3 years ago

Perhaps I mistook how pyright works, but I assumed that pyright needs to be aware of the packages installed in a certain environment and therefore needs to be installed in an environment where all I packages I use/need are installed.

config options venv and venvPath are enough for specifying the virtual environment to use.

I don't know why there is no *lsp-log*, but without further information, it is hard to know what went wrong. Four suggestions: 1. try to make it work locally; 2. install pyright globally; 3. try on toy project; 4. try on the base env.

jingxuanlim commented 3 years ago

@tshu-w, I was able to do (1) and (3) without conda. I installed pyright on my local system and was able to start a server with a local python file. Three new buffers were created: *lsp-log*, *pyright*, and *pyright::stderr*.

However, the problem persisted remotely.

I was not able to do (2) because I don't have sudo on my remote system leading to permission errors.

I guess the next step is to figure out why the *lsp-log* buffer isn't created when viewing a tramp python file. Any ideas?

tshu-w commented 3 years ago

Sorry, no more ideas. The last one to check https://github.com/emacs-lsp/lsp-pyright/issues/10#issuecomment-780739945

jingxuanlim commented 3 years ago

Okay, after consulting with @yyoncho on lsp-mode's discord, I found out that lsp-clients is obsolete. Instead he advised using either lsp-mode or lsp-pyright. Therefore, I switched to using after! lsp-pyright instead of after! lsp-clients. The following config worked for me on doom-emacs, for anyone who's wondering if this was resolved. Thanks also @tshu-w for the help and @mjlbach for the original config!

(setq enable-remote-dir-locals t)
(setq enable-local-variables :all)
(after! tramp
  (add-to-list 'tramp-remote-path 'tramp-own-remote-path))

(use-package! lsp-mode
  :commands lsp
  :hook
  (python-mode . lsp))

(after! lsp-pyright
  (setq lsp-log-io t)
  (setq lsp-pyright-use-library-code-for-types t)
  (setq lsp-pyright-diagnostic-mode "workspace")
  (lsp-register-client
    (make-lsp-client
      :new-connection (lsp-tramp-connection (lambda ()
                                      (cons "pyright-langserver"
                                            lsp-pyright-langserver-command-args)))
      :major-modes '(python-mode)
      :remote? t
      :server-id 'pyright-remote
      :multi-root t
      :priority 3
      :initialization-options (lambda () (ht-merge (lsp-configuration-section "pyright")
                                                   (lsp-configuration-section "python")))
      :initialized-fn (lambda (workspace)
                        (with-lsp-workspace workspace
                          (lsp--set-configuration
                          (ht-merge (lsp-configuration-section "pyright")
                                    (lsp-configuration-section "python")))))
      :download-server-fn (lambda (_client callback error-callback _update?)
                            (lsp-package-ensure 'pyright callback error-callback))
      :notification-handlers (lsp-ht ("pyright/beginProgress" 'lsp-pyright--begin-progress-callback)
                                    ("pyright/reportProgress" 'lsp-pyright--report-progress-callback)
                                    ("pyright/endProgress" 'lsp-pyright--end-progress-callback))))
  )

This is still working sub-optimally for me as I have to run M-x lsp-workspace-restart 2 times before the scan starts and I get completions. I wonder why this is the case...

tshu-w commented 3 years ago

aha, I don't even know lsp-clients.

This is still working sub-optimally for me as I have to run M-x lsp-workspace-restart 2 times before the scan starts and I get completions. I wonder why this is the case...

As mention above, lsp currently not work well with tramp, you should upgrade tramp and change lsp-tramp-connection by yourself.

jingxuanlim commented 3 years ago

@tshu-w, I see! Is my current lsp-tramp-connection, as shown above (copied from OP), sufficient?

tshu-w commented 3 years ago

I think the above pr link https://github.com/emacs-lsp/lsp-mode/pull/2531 has made it clear that you need to modify the lsp-tramp-connection function yourself and upgrade the tramp. You did not show the modified lsp-tramp-connection above.

(defun lsp-tramp-connection (local-command &optional generate-error-file-fn)
    "Create LSP stdio connection named name.
LOCAL-COMMAND is either list of strings, string or function which
returns the command to execute."
    (defvar tramp-connection-properties)
    ;; Force a direct asynchronous process.
    (when (file-remote-p default-directory)
      (add-to-list 'tramp-connection-properties
                   (list (regexp-quote (file-remote-p default-directory))
                         "direct-async-process" t)))
    (list :connect (lambda (filter sentinel name environment-fn)
                     (let* ((final-command (lsp-resolve-final-function
                                            local-command))
                            (_stderr (or (when generate-error-file-fn
                                           (funcall generate-error-file-fn name))
                                         (format "/tmp/%s-%s-stderr" name
                                                 (cl-incf lsp--stderr-index))))
                            (process-name (generate-new-buffer-name name))
                            (process-environment
                             (lsp--compute-process-environment environment-fn))
                            (proc (make-process
                                   :name process-name
                                   :buffer (format "*%s*" process-name)
                                   :command final-command
                                   :connection-type 'pipe
                                   :coding 'no-conversion
                                   :noquery t
                                   :filter filter
                                   :sentinel sentinel
                                   :file-handler t)))
                       (cons proc proc)))
          :test? (lambda () (-> local-command lsp-resolve-final-function
                           lsp-server-present?))))
jdtsmith commented 2 years ago

I wanted to thank the OP for their remote pyright setup code, which is working well. I came up with a few workflow simplifications that might be useful. In terms of paths/virtual environments, if you are frequently using similar remote venvs, there is the simple strategy of adding to your .ssh/config file something like:

Host host-venv
    HostName my.host.name
    RequestTTy force
    RemoteCommand bash --init-file .bash_venv

where the remote file .bash_venv contains . path/to/venv/bin/activate. Then always visit files you intend to use with that venv as /ssh:host-venv:file.py. You could setup a few this way as well.

Another convenience can be had with a small addition like:

(defun my/python-shell-remote-name (name)
    (if-let ((buffer-file-name) (host (file-remote-p buffer-file-name 'host)))
    (format "%s<%s>" name host)
      name))
(advice-add #'python-shell-get-process-name
          :filter-return #'my/python-shell-remote-name)

This gives you a buffer name like *Python<host-venv>* when you invoke run-python (C-c C-p).