search

julia-vscode / LanguageServer.jl

An implementation of the Microsoft Language Server Protocol for the Julia language.
Other
364 stars 79 forks source link

LangaugeServer.jl crashing on a "didChange" request when using with eglot (Emacs) #403

Closed orialb closed 4 years ago

orialb commented 4 years ago

When using with eglot I can reproduce a crash quite consistently by adding a new function definition at the end of the file and moving point to the end statement. That is, in the time of the crash the content of the file I was editing was ("|" denotes cursor position):

function foo()
|end

Here is the stacktrace

ERROR: LoadError: BoundsError: attempt to access 1-element Array{Int64,1} at index [2]
Stacktrace:
 [1] getindex at ./array.jl:729 [inlined]
 [2] get_offset(::LanguageServer.Document, ::Int64, ::Int64) at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/document.jl:59
 [3] get_offset at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/document.jl:66 [inlined]
 [4] get_offset at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/document.jl:67 [inlined]
 [5] applytextdocumentchanges(::LanguageServer.Document, ::LanguageServer.TextDocumentContentChangeEvent) at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/requests/textdocument.jl:241
 [6] process(::LanguageServer.JSONRPC.Request{Val{Symbol("textDocument/didChange")},LanguageServer.DidChangeTextDocumentParams}, ::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/requests/textdocument.jl:128
 [7] run(::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/j2G4Z/src/languageserverinstance.jl:47
 [8] top-level scope at none:0
 [9] include at ./boot.jl:326 [inlined]
 [10] include_relative(::Module, ::String) at ./loading.jl:1038
 [11] include(::Module, ::String) at ./sysimg.jl:29
 [12] exec_options(::Base.JLOptions) at ./client.jl:267
 [13] _start() at ./client.jl:436

and here is the last request sent to the server

client-notification Mon Oct 28 09:12:48 2019:
(:jsonrpc "2.0" :method "textDocument/didChange" :params
          (:textDocument
           (:uri "file:///Users/USERNAME/tmp/LspExample/src/foo.jl" :version 26)
           :contentChanges
           [(:range
             (:start
              (:line 0 :character 14)
              :end
              (:line 0 :character 14))
             :rangeLength 0 :text "\n")
            (:range
             (:start
              (:line 1 :character 0)
              :end
              (:line 1 :character 0))
             :rangeLength 0 :text "    ")
            (:range
             (:start
              (:line 1 :character 4)
              :end
              (:line 1 :character 4))
             :rangeLength 0 :text "e")
            (:range
             (:start
              (:line 1 :character 5)
              :end
              (:line 1 :character 5))
             :rangeLength 0 :text "n")
            (:range
             (:start
              (:line 1 :character 6)
              :end
              (:line 1 :character 6))
             :rangeLength 0 :text "d")
            (:range
             (:start
              (:line 1 :character 0)
              :end
              (:line 1 :character 4))
             :rangeLength 4 :text "")]))

This might be related to eglot because when using lsp-mode I don't see these crashes so far.

I'm using Julia 1.1 and the versions of the related packages are :

[00ebfdb7] CSTParser v0.6.1 #72a9321 (https://github.com/julia-vscode/CSTParser.jl.git)
[ffa9a821] DocumentFormat v0.4.1 #6ccbfaa (https://github.com/julia-vscode/DocumentFormat.jl.git)
[682c06a0] JSON v0.20.0
[2b0e0bc5] LanguageServer v0.6.1 #a91b1ea (https://github.com/julia-vscode/LanguageServer.jl.git)
[b3cc710f] StaticLint v0.2.2 #68a8aa1 (https://github.com/julia-vscode/StaticLint.jl.git)
[cf896787] SymbolServer v0.2.5 #6c4f0ae (https://github.com/julia-vscode/SymbolServer.jl.git)
[0796e94c] Tokenize v0.5.5 #101b8ac (https://github.com/JuliaLang/Tokenize.jl.git)
[30578b45] URIParser v0.4.0+ #7d0dc91 (https://github.com/JuliaWeb/URIParser.jl.git)

Tagging @non-Jedi as per the request in the discourse discussion.

orialb commented 4 years ago

As an extra info that might help in reproducing this, compared to the setup suggested in the wiki I also added this to my emacs config (setq eglot-ignored-server-capabilites '(:hoverProvider)).

I could try to see if the problem persists when I remove this extra config (it might take me some time to get back to this).

non-Jedi commented 4 years ago

Minimum reproducer:

import LanguageServer
const LS = LanguageServer

doc = LS.Document("file:///home/adam/tmp/test.jl", "function foo()", false)
c1 = LS.TextDocumentContentChangeEvent(LS.Range(LS.Position(0,14), LS.Position(0,14)),
                                       0, "\n")
c2 = LS.TextDocumentContentChangeEvent(LS.Range(LS.Position(1,0), LS.Position(1,0)),
                                       0, "    ")
LS.applytextdocumentchanges(doc, c1)
LS.applytextdocumentchanges(doc, c2)

Interestingly, the following does not reproduce:

import LanguageServer
const LS = LanguageServer

doc = LS.Document("file:///home/adam/tmp/test.jl", "function foo()\n", false)
c2 = LS.TextDocumentContentChangeEvent(LS.Range(LS.Position(1,0), LS.Position(1,0)),
                                       0, "    ")
LS.applytextdocumentchanges(doc, c2)

Digging in more, the problem is that the first call to applytextdocumentchanges does not update doc._line_offsets. It's not entirely clear to me where that's supposed to happen, so I can't diagnose why it isn't.

orialb commented 4 years ago

I tried testing the fix in https://github.com/julia-vscode/LanguageServer.jl/pull/405 but I am not able to get the language server working properly with the PR branch. eglot manages to connect to it but then the server is constantly crashing.

If I just update LangaugeServer.jl to the PR branch and keep all the other packages in the versions specified above, I get: 

Process EGLOT (LspExample/julia-mode) stderr finished
[ Info: Started symbol server
[ Info: store set
ERROR: LoadError: UndefVarError: PackageRef not defined
Stacktrace:
 [1] collect_completions(::SymbolServer.ModuleStore, ::String, ::LanguageServer.Range, ::Array{LanguageServer.CompletionItem,1}, ::LanguageServerInstance, ::Bool) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:245
 [2] collect_completions at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:239 [inlined]
 [3] collect_completions(::CSTParser.EXPR, ::String, ::LanguageServer.Range, ::Array{LanguageServer.CompletionItem,1}, ::LanguageServerInstance, ::Bool) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:259
 [4] collect_completions(::CSTParser.EXPR, ::String, ::LanguageServer.Range, ::Array{LanguageServer.CompletionItem,1}, ::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:255
 [5] collect_completions(::CSTParser.EXPR, ::String, ::LanguageServer.Range, ::Array{LanguageServer.CompletionItem,1}, ::LanguageServerInstance, ::Bool) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:264
 [6] collect_completions at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:255 [inlined]
 [7] process(::LanguageServer.JSONRPC.Request{Val{Symbol("textDocument/completion")},LanguageServer.CompletionParams}, ::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:115
 [8] run(::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/languageserverinstance.jl:47
 [9] top-level scope at none:0
 [10] include at ./boot.jl:326 [inlined]
 [11] include_relative(::Module, ::String) at ./loading.jl:1038
 [12] include(::Module, ::String) at ./sysimg.jl:29
 [13] exec_options(::Base.JLOptions) at ./client.jl:267
 [14] _start() at ./client.jl:436
in expression starting at /Users/USERNAME/.spacemacs.d/lisp/eglot-julia/eglot.jl:10

If I just add the PR branch in a clean environment and let the other packages be installed according to the dependencies in LanguageServer.jl I get: 

ERROR: LoadError: MethodError: no method matching Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64, ::Tokenize.Tokens.TokenError, ::Bool)
Closest candidates are:
  Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64, ::Tokenize.Tokens.TokenError, ::Bool, !Matched::Bool) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:70
  Tokenize.Tokens.RawToken(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, !Matched::Any) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:70
  Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:82
Stacktrace:
 [1] get_toks(::LanguageServer.Document, ::Int64) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/utilities.jl:139
 [2] get_partial_completion(::LanguageServer.Document, ::Int64) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:123
 [3] process(::LanguageServer.JSONRPC.Request{Val{Symbol("textDocument/completion")},LanguageServer.CompletionParams}, ::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:24
 [4] run(::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/languageserverinstance.jl:47
 [5] top-level scope at none:0
 [6] include at ./boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1038
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] exec_options(::Base.JLOptions) at ./client.jl:267
 [10] _start() at ./client.jl:436
in expression starting at /Users/USERNAME/.spacemacs.d/lisp/eglot-julia/eglot.jl:10
[ Info: Started symbol server
[ Info: store set
ERROR: LoadError: MethodError: no method matching Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64, ::Tokenize.Tokens.TokenError, ::Bool)
Closest candidates are:
  Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64, ::Tokenize.Tokens.TokenError, ::Bool, !Matched::Bool) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:70
  Tokenize.Tokens.RawToken(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, !Matched::Any) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:70
  Tokenize.Tokens.RawToken(::Tokenize.Tokens.Kind, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}, ::Int64, ::Int64) at /Users/USERNAME/.julia/packages/Tokenize/xA1lh/src/token.jl:82
Stacktrace:
 [1] get_toks(::LanguageServer.Document, ::Int64) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/utilities.jl:139
 [2] get_partial_completion(::LanguageServer.Document, ::Int64) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:123
 [3] process(::LanguageServer.JSONRPC.Request{Val{Symbol("textDocument/completion")},LanguageServer.CompletionParams}, ::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/requests/completions.jl:24
 [4] run(::LanguageServerInstance) at /Users/USERNAME/.julia/packages/LanguageServer/ePE1T/src/languageserverinstance.jl:47
 [5] top-level scope at none:0
 [6] include at ./boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1038
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] exec_options(::Base.JLOptions) at ./client.jl:267
 [10] _start() at ./client.jl:436
in expression starting at /Users/USERNAME/.spacemacs.d/lisp/eglot-julia/eglot.jl:10

where the client request is:

client-request (id:11) Mon Oct 28 19:15:57 2019:
(:jsonrpc "2.0" :id 11 :method "textDocument/completion" :params
          (:textDocument
           (:uri "file:///Users/USERNAME/tmp/LspExample/src/foo.jl")
           :position
           (:line 12 :character 14)
           :context
           (:triggerKind 1)))

What versions of the other dependencies are you using?

non-Jedi commented 4 years ago 
(eglot-julia) pkg> st
    Status `~/.dotfiles/emacs/lisp/eglot-julia/Project.toml`
  [00ebfdb7] CSTParser v0.6.2 #master (https://github.com/julia-vscode/CSTParser.jl.git)
  [ffa9a821] DocumentFormat v0.4.2-DEV #master (https://github.com/julia-vscode/DocumentFormat.jl.git)
  [2b0e0bc5] LanguageServer v0.6.1-DEV #fix-multiple-didChange (https://github.com/non-Jedi/LanguageServer.jl)
  [b3cc710f] StaticLint v0.2.3-DEV #master (https://github.com/julia-vscode/StaticLint.jl.git)
  [cf896787] SymbolServer v1.0.1-DEV #master (https://github.com/julia-vscode/SymbolServer.jl.git)
  [0796e94c] Tokenize v0.5.6
  [6462fe0b] Sockets
orialb commented 4 years ago

Hmm I think I have the same versions as you now but I still keep getting the no method matching Tokenize... error I pasted above.

Maybe you have some of the packages that are marked as #master not on the latest commit? Or can this error be related to a change you made in LagnaugeServer.jl? 

(LspFix) pkg> st
    Status `~/tmp/LspFix/Project.toml`
  [00ebfdb7] CSTParser v0.6.2 #master (https://github.com/julia-vscode/CSTParser.jl.git)
  [ffa9a821] DocumentFormat v0.4.2-DEV #master (https://github.com/julia-vscode/DocumentFormat.jl.git)
  [2b0e0bc5] LanguageServer v0.6.1-DEV #fix-multiple-didChange (https://github.com/non-Jedi/LanguageServer.jl)
  [b3cc710f] StaticLint v0.2.3-DEV #28ad63f (https://github.com/julia-vscode/StaticLint.jl.git)
  [cf896787] SymbolServer v1.0.1-DEV #master (https://github.com/julia-vscode/SymbolServer.jl.git)
  [0796e94c] Tokenize v0.5.6
  [6462fe0b] Sockets
orialb commented 4 years ago

Or can this error be related to a change you made in LagnaugeServer.jl?

I guess it's not related to anything you changed as I've just seen your change is not touching the relevant function in the stack-trace.

orialb commented 4 years ago

Ok, I think I manged to test your PR branch with eglot and it seems to me that your fix solves this issue.

I can't really test this with regular usage because the server seems to keep crashing on completion requests due to https://github.com/julia-vscode/LanguageServer.jl/issues/393. (I tried a lot of different combinations of the latest master branches / commits which are used in the latest vscode plugin with no success). Did you manage to get the server properly functioning with the latest master branches,or some combinations of commits that is able to work with your PR branch?

non-Jedi commented 4 years ago

Did you manage to get the server properly functioning with the latest master branches,or some combinations of commits that is able to work with your PR branch?

It's very odd. It's working fine on my machine with the exact combination of package versions I listed above. I did see the error described in #393 once when I first tried to use the language server after switching to my PR branch, but it has worked flawlessly and without error since then.

EDIT: Nevermind, I'm triggering it now.

non-Jedi commented 4 years ago

I recommend cherry-picking #409. Haven't tested yet personally, but

EDIT: You can use https://github.com/non-Jedi/LanguageServer.jl#all-eglot-fixes until PRs are merged if you like. Everything is working on my end. Alternatively, version 0.5.5 of Tokenize.jl doesn't include https://github.com/JuliaLang/Tokenize.jl/pull/151 which causes the problem, but CSTParser may need the newer version of Tokenize. Not sure.

orialb commented 4 years ago

Thanks a lot for the fixes! I tried your all-eglot-fixes branch (without specifying other dependencies and with specifying other dependencies to certain versions) but the server keeps getting stuck in an infinite PACKAGE failed to load from disk, re-caching loop. This is of course not related to your fixes just to some other dependency that I'm probably not using the right version for.

Anyway I think this issue can be closed now. As for the other bug I'm seeing it seems that there are a lot of version bumps going on for all the dependencies so I'll wait until #405 is merged and try again once thing stabilize a bit (hopefully soon :) ).