Eglot not working with Pyright.

simurgh9 commented 1 year ago

I am on MacOS Venture 13.3.1, GNU Emacs 30.0.50 (brew install emacs-nightly), Eglot 1.14 (found by looking at eglot.el) and finally Pyright 1.1.307 (pip3 install pyright).

My issue is similar to this: https://github.com/joaotavora/eglot/issues/768

As soon as I run eglot, I get,

[eglot] Connected! Server `EGLOT (Doodles/(python-mode python-ts-mode))' now managing `(python-mode python-ts-mode)' buffers in project `Doodles'.
[jsonrpc] Server exited with status 1
[eglot] (warning) Not auto-reconnecting, last one didn't last long.

Here is the *Backtrace*:

Debugger entered--Lisp error: (file-notify-error "File watching not possible, no file descriptor left" 975)
  kqueue-add-watch("/Users/tfn/Dropbox/fortochka/Python/Doodles/.venv/..." (create delete write extend rename) file-notify--callback-kqueue)
  file-notify--add-watch-kqueue("/Users/tfn/Dropbox/fortochka/Python/Doodles/.venv/..." "/Users/tfn/Dropbox/fortochka/Python/Doodles/.venv/..." (change))
  file-notify-add-watch("/Users/tfn/Dropbox/fortochka/Python/Doodles/.venv/..." (change) #f(compiled-function (event) #<bytecode 0x16748f7aa78fa4d8>))
  #f(compiled-function (arg1 arg2 arg3 &rest rest) "Handle dynamic registration of workspace/didChangeWatchedFiles." #<bytecode -0x1e7f6fdb54fcf73a>)(#<eglot-lsp-server eglot-lsp-server-ca08aa04> workspace/didChangeWatchedFiles "55a2d263-0c17-4819-81c9-c6040637c61b" :watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)])
  apply(#f(compiled-function (arg1 arg2 arg3 &rest rest) "Handle dynamic registration of workspace/didChangeWatchedFiles." #<bytecode -0x1e7f6fdb54fcf73a>) #<eglot-lsp-server eglot-lsp-server-ca08aa04> workspace/didChangeWatchedFiles ("55a2d263-0c17-4819-81c9-c6040637c61b" :watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))
  eglot-register-capability(#<eglot-lsp-server eglot-lsp-server-ca08aa04> workspace/didChangeWatchedFiles "55a2d263-0c17-4819-81c9-c6040637c61b" :watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)])
  apply(eglot-register-capability #<eglot-lsp-server eglot-lsp-server-ca08aa04> workspace/didChangeWatchedFiles "55a2d263-0c17-4819-81c9-c6040637c61b" (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))
  eglot--register-unregister(#<eglot-lsp-server eglot-lsp-server-ca08aa04> [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))] register)
  #f(compiled-function (arg1 arg2 &rest rest) "Handle server request client/registerCapability." #<bytecode -0x1aa987436e2ee13c>)(#<eglot-lsp-server eglot-lsp-server-ca08aa04> client/registerCapability :registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))])
  apply(#f(compiled-function (arg1 arg2 &rest rest) "Handle server request client/registerCapability." #<bytecode -0x1aa987436e2ee13c>) #<eglot-lsp-server eglot-lsp-server-ca08aa04> client/registerCapability (:registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))]))
  eglot-handle-request(#<eglot-lsp-server eglot-lsp-server-ca08aa04> client/registerCapability :registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))])
  apply(eglot-handle-request #<eglot-lsp-server eglot-lsp-server-ca08aa04> client/registerCapability (:registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))]))
  #f(compiled-function (server method params) #<bytecode -0xab4db0d03eebf61>)(#<eglot-lsp-server eglot-lsp-server-ca08aa04> client/registerCapability (:registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))]))
  jsonrpc-connection-receive(#<eglot-lsp-server eglot-lsp-server-ca08aa04> (:jsonrpc "2.0" :id 4 :method "client/registerCapability" :params (:registrations [(:id "55a2d263-0c17-4819-81c9-c6040637c61b" :method "workspace/didChangeWatchedFiles" :registerOptions (:watchers [(:globPattern "**/pyrightconfig.json" :kind 7) (:globPattern "**" :kind 7)]))])))
  jsonrpc--process-filter(#<process EGLOT (Doodles/(python-mode python-ts-mode))> "Content-Length: 100\15\n\15\n{\"jsonrpc\":\"2.0\",\"method\":\"...")

And these are the event logs,

[internal] Fri May 12 02:33:40 2023:
(:message "Running language server: /Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles/.venv/bin/pyright-langserver --stdio")
[client-request] (id:1) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 1 :method "initialize" :params
  (:processId 13391 :clientInfo
    (:name "Eglot")
    :rootPath "/Users/tfn/Dropbox/fortochka/Python/Doodles/" :rootUri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles" :initializationOptions #s(hash-table size 1 test eql rehash-size 1.5 rehash-threshold 0.8125 data
                                                                                                                                                                           ())
    :capabilities
    (:workspace
      (:applyEdit t :executeCommand
        (:dynamicRegistration :json-false)
        :workspaceEdit
        (:documentChanges t)
        :didChangeWatchedFiles
        (:dynamicRegistration t)
        :symbol
        (:dynamicRegistration :json-false)
        :configuration t :workspaceFolders t)
      :textDocument
      (:synchronization
        (:dynamicRegistration :json-false :willSave t :willSaveWaitUntil t :didSave t)
        :completion
        (:dynamicRegistration :json-false :completionItem
          (:snippetSupport :json-false :deprecatedSupport t :resolveSupport
            (:properties
              ["documentation" "details" "additionalTextEdits"])
            :tagSupport
            (:valueSet
              [1]))
          :contextSupport t)
        :hover
        (:dynamicRegistration :json-false :contentFormat
          ["markdown" "plaintext"])
        :signatureHelp
        (:dynamicRegistration :json-false :signatureInformation
          (:parameterInformation
            (:labelOffsetSupport t)
            :documentationFormat
            ["markdown" "plaintext"]
            :activeParameterSupport t))
        :references
        (:dynamicRegistration :json-false)
        :definition
        (:dynamicRegistration :json-false :linkSupport t)
        :declaration
        (:dynamicRegistration :json-false :linkSupport t)
        :implementation
        (:dynamicRegistration :json-false :linkSupport t)
        :typeDefinition
        (:dynamicRegistration :json-false :linkSupport t)
        :documentSymbol
        (:dynamicRegistration :json-false :hierarchicalDocumentSymbolSupport t :symbolKind
          (:valueSet
            [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26]))
        :documentHighlight
        (:dynamicRegistration :json-false)
        :codeAction
        (:dynamicRegistration :json-false :codeActionLiteralSupport
          (:codeActionKind
            (:valueSet
              ["quickfix" "refactor" "refactor.extract" "refactor.inline" "refactor.rewrite" "source" "source.organizeImports"]))
          :isPreferredSupport t)
        :formatting
        (:dynamicRegistration :json-false)
        :rangeFormatting
        (:dynamicRegistration :json-false)
        :rename
        (:dynamicRegistration :json-false)
        :inlayHint
        (:dynamicRegistration :json-false)
        :publishDiagnostics
        (:relatedInformation :json-false :codeDescriptionSupport :json-false :tagSupport
          (:valueSet
            [1 2])))
      :window
      (:workDoneProgress t)
      :general
      (:positionEncodings
        ["utf-32" "utf-8" "utf-16"])
      :experimental #s(hash-table size 1 test eql rehash-size 1.5 rehash-threshold 0.8125 data
                        ()))
    :workspaceFolders
    [(:uri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles" :name "~/Dropbox/fortochka/Python/Doodles/")]))
[server-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "window/logMessage" :params
  (:type 3 :message "Pyright language server 1.1.307 starting"))
[server-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "window/logMessage" :params
  (:type 3 :message "Server root directory: /Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/"))
[server-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "window/logMessage" :params
  (:type 3 :message "Starting service instance \"~/Dropbox/fortochka/Python/Doodles/\""))
[server-reply] (id:1) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 1 :result
  (:capabilities
    (:textDocumentSync 2 :definitionProvider
      (:workDoneProgress t)
      :declarationProvider
      (:workDoneProgress t)
      :typeDefinitionProvider
      (:workDoneProgress t)
      :referencesProvider
      (:workDoneProgress t)
      :documentSymbolProvider
      (:workDoneProgress t)
      :workspaceSymbolProvider
      (:workDoneProgress t)
      :hoverProvider
      (:workDoneProgress t)
      :documentHighlightProvider
      (:workDoneProgress t)
      :renameProvider
      (:prepareProvider t :workDoneProgress t)
      :completionProvider
      (:triggerCharacters
        ["." "[" "\"" "'"]
        :resolveProvider t :workDoneProgress t :completionItem
        (:labelDetailsSupport t))
      :signatureHelpProvider
      (:triggerCharacters
        ["(" "," ")"]
        :workDoneProgress t)
      :codeActionProvider
      (:codeActionKinds
        ["quickfix" "source.organizeImports"]
        :workDoneProgress t)
      :executeCommandProvider
      (:commands
        []
        :workDoneProgress t)
      :callHierarchyProvider t :workspace
      (:workspaceFolders
        (:supported t :changeNotifications t)))))
[client-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "initialized" :params #s(hash-table size 1 test eql rehash-size 1.5 rehash-threshold 0.8125 data
                                                  ()))
[client-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "textDocument/didOpen" :params
  (:textDocument
    (:uri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles/tree.py" :version 0 :languageId "python" :text "import numpy as np\n\nnp.random.seed(0)\n\ndata = np.loadtxt('tennis.csv', delimiter=',', skiprows=1)\n")))
[client-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "workspace/didChangeConfiguration" :params
  (:settings #s(hash-table size 1 test eql rehash-size 1.5 rehash-threshold 0.8125 data
                 ())))
[server-request] (id:0) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 0 :method "client/registerCapability" :params
  (:registrations
    [(:id "76627e0b-0bc0-4d39-9f6c-c5e7cf93c62c" :method "workspace/didChangeWatchedFiles" :registerOptions
       (:watchers
         [(:globPattern "**/pyrightconfig.json" :kind 7)
           (:globPattern "**" :kind 7)]))]))
[client-reply] (id:0) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 0 :result nil)
[server-request] (id:1) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 1 :method "workspace/configuration" :params
  (:items
    [(:scopeUri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles" :section "python")]))
[client-reply] (id:1) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 1 :result
  [nil])
[server-request] (id:2) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 2 :method "workspace/configuration" :params
  (:items
    [(:scopeUri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles" :section "python.analysis")]))
[client-reply] (id:2) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 2 :result
  [nil])
[server-request] (id:3) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 3 :method "workspace/configuration" :params
  (:items
    [(:scopeUri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles" :section "pyright")]))
[client-reply] (id:3) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 3 :result
  [nil])
[server-notification] Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :method "window/logMessage" :params
  (:type 3 :message "Found 11 source files"))
[server-request] (id:4) Fri May 12 02:33:40 2023:
(:jsonrpc "2.0" :id 4 :method "client/registerCapability" :params
  (:registrations
    [(:id "5f68e7c3-d6be-46f9-b8f5-18ca46acd165" :method "workspace/didChangeWatchedFiles" :registerOptions
       (:watchers
         [(:globPattern "**/pyrightconfig.json" :kind 7)
           (:globPattern "**" :kind 7)]))]))
[client-reply] (id:4) ERROR Fri May 12 02:33:41 2023:
(:jsonrpc "2.0" :id 4 :error
  (:code -32603 :message "Internal error"))
[server-notification] Fri May 12 02:33:41 2023:
(:jsonrpc "2.0" :method "textDocument/publishDiagnostics" :params
  (:uri "file:///Users/tfn/Library/CloudStorage/Dropbox/fortochka/Python/Doodles/tree.py" :version 0 :diagnostics
    []))
[stderr] Error: Internal error
[stderr]     at handleResponse (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:565:48)
[stderr]     at handleMessage (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:345:13)
[stderr]     at processMessageQueue (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:362:17)
[stderr]     at Immediate.<anonymous> (/Users/tfn/.cache/pyright-python/1.1.307/node_modules/pyright/dist/pyright-internal/node_modules/vscode-jsonrpc/lib/common/connection.js:334:13)
[stderr]     at processImmediate (node:internal/timers:478:21)
[internal] Fri May 12 02:33:41 2023:
(:message "Connection state changed" :change "exited abnormally with code 1\n")

----------b---y---e---b---y---e----------
[stderr] 
[stderr] 
[stderr] nil
[stderr] nil
[stderr] Process EGLOT (Doodles/(python-mode python-ts-mode)) stderr finished

I tried googling the no file descriptors left error and found this,

Some language servers used by lsp-mode start watching changes to every file in a project. I happen to work with a big project in which the only option has been to disable lsp-mode due to this.

Well, but I am not using lsp-mode but maybe something similar is happening with eglot?

Any hint to a resolution is appreciated.

EDIT

Okay I've figured something out, this only happens when I run eglot from within a file that is in the same directory as a virtual environment folder, e.g., .venv. I am guessing that happens because emacs tries to create a watch for all the files in there. I am not sure what a good fix is though other than moving the .venv away.

joaotavora commented 1 year ago

When creating this issue, did the following text appear in front of you?

<!-- To continue,  you will need to provide some elements
including a description of a [MINIMUM REPRODUCIBLE EXAMPLE][1] (MRE).  

The instructions on HOW to report MRE bugs can be found here:

  https://joaotavora.github.io/eglot/#Troubleshooting-Eglot

Here are GOOD examples of recent MRE bug reports:

  https://github.com/joaotavora/eglot/issues/1062
  https://github.com/joaotavora/eglot/issues/1037
  https://github.com/joaotavora/eglot/issues/940

If you don't have those elements, please DO NOT, I repeat, DO NOT
CREATE AN BUG REPORT ISSUE. Creating an incomplete bug report issue
DOES NOT bring more attention to your problem, in fact quite the
contrary.

joaotavora commented 1 year ago

I am guessing that happens because emacs tries to create a watch for all the files in there.

And I am guessing too, because I don't have an MRE. Maybe Eglot could be taught by the user not to watch certain folder with very large contents, or maybe you need to explain what the structure of your project and use project.el to tell Eglot that the project starts slightly deeper in the hierarchy whereit that doesn't include so million files. But there's no MRE to reproduce this.

simurgh9 commented 1 year ago

Is this MRE?

mre

The LSP mode has this solution for the same problem.

simurgh9 commented 1 year ago

@joaotavora ?

joaotavora commented 1 year ago

simurgh9 commented 1 year ago

By golly man, did you read that link yourself?

Yeah I know I did not give all that was asked (because I was confused myself) at first but what from that link in the red box is missing from the screenshot I responded with?

mkdir mre
cd mre
python3 -m venv .venv
source .venv/bin/activate # maybe different depending upon the OS
pip3 install pyright numpy numpydoc
emacs test.py -Q -f toggle-debug-on-error -f eglot

Does this not replicate the issue for you?

If you think you have found a bug, we want to hear about it.

Does not look like it. I thought Emacs community was friendlier to the first time bug reporters, maybe the Node under the Pyright's hood is rubbing off.

joaotavora commented 1 year ago

By golly man, did you read that link yourself?

Not only did I read it, I wrote it. Because I don't want to write it every time I am faced with a similar difficulties.

Does not look like it. I thought Emacs community was friendlier to the first time bug reporters, maybe the Node under the Pyright's hood is rubbing off.

I'm sorry you find this unfriendly. All the times it takes to you to read the information I wrote is negligible to the effort I put in writing it again and again and guessing your setup. Many other projects take a similar stance. See for example this link where I modelled some of Eglot's bug reporting practice.

If this seem unfriendly to you, you should try put yourself in the shoes of a a single unpaid software maintainer of a popular package dealing with many bug reports (and still trying to have a dayjob and a non-programming-related life).

Does this not replicate the issue for you?

Sigh. I won't even lose time trying it until you specify what test.py is and what versions of the software (Emacs in this case) you are using. But I'll say you're getting closer with this recipe though, which is a good sign.

simurgh9 commented 1 year ago

Sigh. I won't even lose time trying it until you specify what test.py is

When you run emacs anyfilewhatsoever.extension, does it matter what the contents of the file are? If it does not exist, it creates one for you. test.py is the new file we are creating, the contents do not matter in the reproduction of this error.

The issue, as I tried to guess before, is that there are too many files in the directory .venv after pip3 install pyright numpy numpydoc. So when we launch Pyright à la eglot, emacs tries to create too many file watches. The contents (or a lack thereof) of the current file do not matter.

I created the mre directory mkdir mre, right? and then cd mre into it? So it must be empty right? The 'M' in the MRE stands for minimal.

If this seem unfriendly to you, you should try put yourself in the shoes of a a single unpaid software maintainer of a popular package dealing with many bug reports (and still trying to have a dayjob and a non-programming-related life).

I think the description does not prompt for a change of shoes as mine already fit fine ;). I appreciate what you do. I get it, but do you get why,

Sigh. I won't even lose time trying it until you specify what test.py

You would not have lost time if you had not automatically made the assumption that trying would cause you to lose time?

what versions of the software (Emacs in this case) you are using

In the words of a great friend, did the following text appear in front of you? :)

Perhaps sometimes we are all guilty of expecting pedagogical kindness over reading.

joaotavora commented 1 year ago

When you run emacs anyfilewhatsoever.extension, does it matter what the contents of the file are?

Yes, it usually does. Emacs and Eglot exhibit different behaviour depending on the contents of the file.

In the words of a great friend, did the following text appear in front of you? :)

Sorry, that was in the beginning of the issue. I'd rather have a compact MRE where I don't have to scroll up and forth. You could also use emacs --version and clear it up. Please give compact recipes and thanks in advance.

You would not have lost time if you had not automatically made the assumption that trying would cause you to lose time?

Please put in the work for following the isntructions to supply a good bug report. If you don't want to put this time, you can informally report a problem in the discussions instead. This is also stated in the text that you skipped.

simurgh9 commented 1 year ago

I don't see a form that I can fill out and I think being handed a shell script that literally produces the backtrace is a good bug report. I have read your instructions like 3 times and IDK what else do you want. emacs --version returns

emacs --version
GNU Emacs 30.0.50
Copyright (C) 2023 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

IDK how that clears things up any further.

Please give compact recipes and thanks in advance.

mkdir mre
cd mre
python3 -m venv .venv
source .venv/bin/activate # maybe different depending upon the OS
pip3 install pyright numpy numpydoc
emacs test.py -Q -f toggle-debug-on-error -f eglot

What else do you want!?

Honestly, forget about it if you still don't have everything you need.

joaotavora commented 1 year ago

What else do you want!?

I think I have everything, thanks.

source .venv/bin/activate # maybe different depending upon the OS

I'm on Linux and installing all this stuff will have to be done in a container possibly, but I should find time to look at it this week.

simurgh9 commented 1 year ago

I'm on Linux and installing all this stuff will have to be done in a container possibly, but I should find time to look at it this week.

The source .venv/bin/activate works on Linux and MacOS the same. It's a little different on Windows.

If you are on Linux, likely Python 3 is installed, if it is, then Pip is installed as well. You don't need anything else. The command, python3 -m venv .venv creates a virtual environment. After sourcing, pip3 install pyright numpy numpydoc installs the packages in that virtual environment. So it should not mess up any Python stuff on your system.

joaotavora commented 1 year ago

Can't reproduce on linux. Googling brought me here: https://www.reddit.com/r/emacs/comments/x4p7mg/fix_annoying_max_open_files_for_emacs/

You could also try explaining to pyright that you don't want the contents of the full hidden .venv directory watched. Or switch to linux.

illia-danko commented 1 year ago

@simurgh9 when this LSP provider was added to elgot it supposed to use https://github.com/microsoft/pyright, please try to install it via npm or yarn: npm install -g pyright

joaotavora commented 1 year ago

@illia-danko thanks, but I don't know if that will make a difference.

I think the main problem here is that Pyright tries to watch too many files. Specifically, the files under foo/.venv/... probably don't need any watching because the user isn't supposed to change those files as she is working in other foo/*.py files.

This isn't a real problem on Linux because watching a directory with many files doesn't require many file descriptors. On MacOSX, it does and there aren't usually enough of them.

What I meant by "explaining to pyright that we don't want the contents of the full hidden .venv directory watched" is that maybe there is a Pyright command-line option to achieve that. Or maybe a ticket should be raised with the server to request this.

Eglot could also add code to stop watching .venv, but Eglot (and LSP) is supposed to be server- and language-agnostic. Since "virtual envs" are a Python thing, it should be a server side fix, not an Eglot fix.

joaotavora commented 1 year ago

Seems that this was already requested in Pyright, and the response wasn't good (IMO)

https://github.com/microsoft/pyright/issues/2108#issuecomment-884932599 https://github.com/microsoft/pylance-release/issues/3406

I'll add a comment there asking them to reconsider

simurgh9 commented 1 year ago

So I thought it maybe useful to see what happens when Pyright is ran from the command-line without eglot/emacs. See above.

mkdir mre
cd mre
python3 -m venv .venv
source .venv/bin/activate # maybe different depending upon the OS
pip3 install pyright numpy numpydoc
touch test.py
python3 -m pyright --watch --verbose

The server complains but does not crash.

Search paths for /Users/tfn/Desktop/mre
  /Users/tfn/.cache/pyright-python/1.1.310/node_modules/pyright/dist/typeshed-fallback/stdlib
  /Users/tfn/Desktop/mre
  /Users/tfn/Desktop/mre/typings
  /Users/tfn/.cache/pyright-python/1.1.310/node_modules/pyright/dist/typeshed-fallback/stubs/...
  /opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11
  /opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload
  /Users/tfn/Desktop/mre/.venv/lib/python3.11/site-packages
Watcher could not use native fsevents library. File system watcher disabled.
Adding fs watcher for directories:
 /Users/tfn/Desktop/mre
Watcher could not use native fsevents library. File system watcher disabled.
Found 1 source file
pyright 1.1.310
0 errors, 0 warnings, 0 informations 
Watching for file changes...

joaotavora commented 1 year ago

There is some misunderstanding here @simurgh9 . Maybe Pyright, which is an LSP server, can watch files itself, but usually servers requests clients to do the file watching on their behalf. And, as far as I can understand, that's what's happening in this case.

simurgh9 commented 1 year ago

To be very honest, I am not very familiar with what file watches are. I mean I could guess from the discussion over at the microsoft repo but that's about all.

Maybe Pyright, which is an LSP server, can watch files itself, but usually servers requests clients to do the file watching on their behalf. And, as far as I can understand, that's what's happening in this case.

I see, well but then if Pyright can watch files itself then why are we seeing the error from the client?

joaotavora commented 1 year ago

I see, well but then if Pyright can watch files itself then why are we seeing the error from the client?

I don't know for sure that it can, but your log messages seems to at least indicate that it tires to.

We are seeing errors because it is also asking the client to watch a zillion files. And the client eglot uses Emacs which in turn uses Mac OSX kqueue, and that hits a limit. Thus the client answers with "internal error" and then Pyright seemingly freaks out and also kills itself.

simurgh9 commented 1 year ago

BTW just to avoid any miscommunication, I don't think I am on MacOS X. If I understand correctly, that is MacOS 10, (X being a roman numeral). I am on MacOS Venture 13.3.1.

joaotavora commented 1 year ago

BTW just to avoid any miscommunication, I don't think I am on MacOS X. If I understand correctly, that is MacOS 10, (X being a roman numeral). I am on MacOS Venture 13.3.1.

Force of habit., sorry. They dropped the X some years ago. I last used used MacOS in those versions (when it was a half-decent OS). Doesn't make a difference I think

joaotavora commented 1 year ago

After analysing this and seeing there's little intention to change things over at Pyright, I can recommend the following fixes:

Add a method to eglot-client-capabilities that checks if the current server is Pyright and removes the workspace/didChangeWatchedFiles capability from the list of client capabilities advertised to the server. This is not particularly easy to do.
Teach project.el that .venv is not a directory of interest, i.e. that it is not a "project directory". You can probably use project-vc-ignores for that (read its docstring). It will only work with some kind of version-controlled projects (so not your MRE, unless you add a git init in there.

The latter solution seems to be much easier

(add-to-list 'project-vc-ignores "./.venv/")

You'll lose some file-watching capabilities in .venv ,but by Pyright author's admission, it isn't particularly useful anyway.

joaotavora commented 1 year ago

Also converting this to a discussion which is what it should have been in the first place.

joaotavora / eglot

Eglot not working with Pyright. #1224