SymbolServer.jl can't find `SHA`

[Seelengrab]

Hi,

as mentioned in https://github.com/julia-vscode/LanguageServer.jl/issues/987, my symbol server for a language server can't seem to find SHA, crashing and thus making all symbols missing references. I'm on 1.7-rc1, if that matters.

MWE:

import Pkg
Pkg.activate(;temp=true)
Pkg.add("SymbolServer")
using SymbolServer
srv = SymbolServer.SymbolServerInstance()
s = getstore(srv, <a valid julia environment that doesn't have SHA in its dependencies>)

will result in:

(:failure, IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=305, maxsize=Inf, ptr=306, mark=-1))

julia> msg = String(take!(s[2]))
"ERROR: LoadError: ArgumentError: Package SHA not found in current path:\n- Run `import Pkg; Pkg.add(\"SHA\")` to install the SHA package.\n\nStacktrace:\n [1] require(into::Module, mod::Symbol)\n   @ Base ./loading.jl:967\nin expression starting at /home/sukera/.julia/packages/SymbolServer/rrlGM/src/server.jl:1\n"

Running the following:

julia> LOAD_PATH
3-element Vector{String}:
 "@"
 "@v#.#"
 "@stdlib"

julia> using SHA
 │ Package SHA not found, but a package named SHA is available from a registry.
 │ Install package?
 │   (jl_1io4U8) pkg> add SHA
 └ (y/n) [y]:

So I guess SHA is no longer available for just using in 1.7..?

Possibly related: https://github.com/JuliaLang/julia/pull/41370

[pfitzseb]

That sounds like your Manifest is invalid (created under 1.6, used with 1.7), no?

[Seelengrab]

Which one? The one I Pkg.activate(;temp=true)ed or of my project..? I get the Package SHA not found message in the temporary environment created with 1.7.

Note that my project was never supposed to depend on SHA. On top of that, the project I'm trying to use with the LanguageServer/SymbolServer here doesn't even have a Manifest.toml inside of it - it's basically a fresh clone that's not even instantiated.

[Seelengrab]

I guess the MWE for the SHA problem would be this:

(@v1.7) pkg> activate --temp
  Activating new project at `/tmp/jl_slkzfp`

julia> using SHA
 │ Package SHA not found, but a package named SHA is available from a registry.
 │ Install package?
 │   (jl_slkzfp) pkg> add SHA
 └ (y/n) [y]: ^C
ERROR: ArgumentError: Package SHA not found in current path:
- Run `import Pkg; Pkg.add("SHA")` to install the SHA package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:967

which really should be an issue in julia itself instead of SymbolServer, if that's the root cause...

[pfitzseb]

Hm, that works fine for me:

λ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-rc1 (2021-09-12)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

(@v1.7) pkg> activate --temp
  Activating new project at `/tmp/jl_TijudQ`

julia> using SHA

julia> SHA
SHA

Are you by chance re-using the same global environment for 1.6 and 1.7?

[Seelengrab]

No, I don't even have 1.6 installed on this machine.. This install is basically just a local build of a fresh 1.7-rc1 clone. There never was 1.6 on this machine.

julia> versioninfo()
Julia Version 1.7.0-rc1
Commit 9eade6195e (2021-09-12 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

This is

Linux 5.14.8-arch1-1 #1 SMP PREEMPT Sun, 26 Sep 2021 19:36:15 +0000 x86_64 GNU/Linux

with julia build locally, not from pacman (there were some precompile statements that didn't run during compilation, but I never checked which).

[pfitzseb]

It's possible your build is broken then. I just now rebuilt my RC1 and deleted the global env, and can't repro your issue:

λ julia           
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.7.0-rc1 (2021-09-12)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml` (empty project)

(@v1.7) pkg> activate --temp
  Activating new project at `/tmp/jl_ROIFIg`

julia> using SHA

julia> versioninfo()
Julia Version 1.7.0-rc1
Commit 9eade6195e (2021-09-12 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

[Seelengrab]

I have my doubts that rebuilding 1.7-rc1 would fix this since I didn't change anything from the base image, but I guess I can try? I'll try clearing out the default Manifest first.

[Seelengrab]

Clearing out the manifest didn't fix it:

(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml` (empty project)

julia> using SHA
 │ Package SHA not found, but a package named SHA is available from a registry.
 │ Install package?
 │   (@v1.7) pkg> add SHA
 └ (y/n) [y]: ^C
ERROR: ArgumentError: Package SHA not found in current path:
- Run `import Pkg; Pkg.add("SHA")` to install the SHA package.

Stacktrace:
 [1] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:967

(@v1.7) pkg> activate --temp
  Activating new project at `/tmp/jl_22bbKm`

julia> using SHA
 │ Package SHA not found, but a package named SHA is available from a registry.
 │ Install package?
 │   (jl_22bbKm) pkg> add SHA
 └ (y/n) [y]:

I'll try to rebuild.

[Seelengrab]

Notably, using other Stdlibs that depend on SHA (e.g. LibGit2) worked fine..

[pfitzseb]

Might also be worth trying the official build.

[Seelengrab]

Ok, with a fresh build of 1.7-rc1 it works, which gave me an idea. I recently opened a PR for some doc change (https://github.com/JuliaLang/julia/pull/42454), which was based on master. I didn't switch back to the 1.7-rc1 branch though, to keep the doc change around. Rebuilding forced me to switch back to the "real" 1.7-rc1 branch. Going back to the master branch on the 1.7-rc1 build also failed, so I guess this had to do with that.

I'll try building master to see if it fails there as well as checking whether my language server issues are solved by just being on a pure 1.7-rc1.

[pfitzseb]

(@v1.8) pkg> st
Status `~/.julia/environments/v1.8/Project.toml` (empty project)

(@v1.8) pkg> activate --temp
  Activating new project at `/tmp/jl_GDhN89`

julia> using SHA

julia> versioninfo()
Julia Version 1.8.0-DEV.641
Commit ddd8301599 (2021-10-03 20:49 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

[Seelengrab]

OK, I can confirm that the SHA error is gone - but the Missing references are still there :/

[Seelengrab]

julia> s = getstore(srv, "/home/sukera/projects/BigNums.jl/");

julia> s[2]
Dict{Symbol, SymbolServer.ModuleStore} with 3 entries:
  :Main => ModuleStore(VarRef(nothing, :Main), Dict{Symbol, Any}(), "", true, Symbol[], Symbol[])
  :Base => ModuleStore(VarRef(nothing, :Base), Dict{Symbol, Any}(:range_step_stop_length=>FunctionStore(VarRef(VarRef(nothing, :Base), :rang…
  :Core => ModuleStore(VarRef(nothing, :Core), Dict{Symbol, Any}(:ReadOnlyMemoryError=>DataTypeStore(FakeTypeName(VarRef(VarRef(nothing, :Co…

These seem to be the only modules it can find - am I using getstore wrong?

[pfitzseb]

IIRC you need an instantiated environment or we'll fall back to your (presumably empty) global env.

[Seelengrab]

Hmm that does give me the symbols of dependencies, but not of the module that's defined at that path :/ Do I have to add that specially?

[Seelengrab]

I'm guessing I have to use a temporary environment, where I add Revise and the package I'm developing and use that for the symbol server environment..? Seems like a very roundabout way of doing it, but I guess that's the only way.. I think that would also pollute the symbol server cache with Revise stuff though :/

[Seelengrab]

Ok, reading through https://github.com/julia-vscode/LanguageServer.jl/issues/748 it seems what I originally wanted to do just isn't possible right now outside of VSCode. I'll add my thoughts in regards to that there.