GemNotFoundException when using solargraph as a Ruby LSP

cuducos commented 2 years ago

I ran into an issue where Ruby LSP with solargraph was working and then, suddenly, it was failing with in ~/.cache/nvim/lsp.log:

[ ERROR ] 2021-10-21T09:59:03-0400 ] ...llar/neovim/0.5.0/share/nvim/runtime/lua/vim/lsp/rpc.lua:462 ]  "rpc"   "solargraph"    "stderr"    "/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:283:in `find_spec_for_exe': can't find gem solargraph (>= 0.a) with executable solargraph (Gem::GemNotFoundException)\n\tfrom /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems.rb:302:in `activate_bin_path'\n\tfrom /usr/local/bin/solargraph:23:in `<main>'\n"

That was awkward since:

why would it use the OS's Ruby?
I haven't changed a thing in the LSP setting…

After a while, I could figure out what has happened — so I’m registering this issue here just in case someone stumbles at the it too!

Context

:LspInstall solargraph creates a binary at ~/.local/share/nvim/lsp_servers/solargraph/bin — this is a Ruby file with a shebang. In my case, that pointed to #!/opt/rubies/2.7.2/bin/ruby. I’m not sure why nvim -lsp-installer used that particular Ruby version, but my guess is that it might be the PATH handling done by chroot (the tool I use to handle different Ruby versions locally).

I don't remember in which project/directory I first run :LspInstall solargraph, so I’m not sure what version of Ruby the Gem was installed into.

TLDR

I just got the path from the ~/.local/share/nvim/lsp_servers/solargraph/bin's shebang and installed solargraph Gem there. In my case:

:!/opt/rubies/2.7.2/bin/gem install solargraph

williamboman commented 2 years ago

Hey not sure if you intentionally closed this immediately for future travelers to find or if it was by accident. To be honest I have no idea how RubyGems works under the hood - I just tried on one of my systems and it seems to set a #!/usr/bin/ruby2.7 shebang. This is how it currently installs gems - do you see anything that could be improved?

cuducos commented 2 years ago

I'm also a noob in how Ruby and RubyGems are handled, so I'm really not sure if is there a way to improve.

Honestly, I opened (and closed) the issue mostly to share a solution with other ~~issues~~ users that may face the same problem. I wasn't really thinking of an improvement… but if any Rubyist out there happens to have a better idea in mind, maybe you could re-open the issue, right?

calvh commented 2 years ago

Thank you for sharing your solution. I faced the same issue (using asdf). I installed solargraph in the system directory instead as a stopgap measure.

williamboman commented 2 years ago

Do you have any ideas how this can be dealt with? To me it sounds like the package manager is hardcoding the full executable path of whatever Ruby version is currently active? Is there any specific reason it's doing so?

Slotos commented 2 years ago

Can an interface be provided to run commands with LspInstaller env? Not even in the background, a :terminal window will be most useful.

Right now I'm seeing a similar issue, with gem environment set for LSP server not having anything to actually be a useful server. Easily fixed with GEM_HOME=".local/share/nvim/lsp_servers/solargraph" GEM_PATH=".local/share/nvim/lsp_servers/solargraph" bundle install, but getting those paths was a journey.

williamboman commented 2 years ago

Can an interface be provided to run commands with LspInstaller env? Not even in the background, a :terminal window will be most useful.

Hmm what's the use case? This would otherwise be fairly easy to accomplish, either as a core capability or through the existing APIs (e.g., through a custom command that makes use of the .root_dir property of the :h nvim-lsp-installer.Server instance). Worth noting is that each gem-based server has its own environment, so having one single shared one is currently not possible. I'd be happy to change this somehow, but I don't particularly use Ruby version managers nor do I suffer from this problem so I really don't know how it could be improved atm.

Slotos commented 2 years ago

My use case

Solargraph - likely depending on configuration, I hadn't dug too deep here - needs to load at least a portion of application gems in order to work. In my case, rubocop and a number of support gems were missing, leading to GemNotFoundException being thrown. After installing a few of them one by one, GEM_HOME="$HOME/.local/share/nvim/lsp_servers/solargraph" GEM_PATH="$HOME/.local/share/nvim/lsp_servers/solargraph" bundle install solved the issue. I will likely need to run it again after major updates to Gemfile.

Additionally, access to lsp-installer environment is quite useful for installing solargraph plugins.

Ruby versions

Native extensions might not work across different ruby versions. This can be remedied by adding ruby version to gem-based servers' GEM_ paths. A case of older ruby versions becoming incompatible with newer language server versions can be solved with custom servers.

For every application and every ruby version, users will need to install their gems multiple times: once for application itself, once for every language server. But as something that can be automated with a little Lua or Vimscript, it doesn't bother me.

srcrip commented 2 years ago

The answer from @Slotos was the missing piece for me as well, if it wasn't clear for anyone else reading you can just do:

GEM_HOME="$HOME/.local/share/nvim/lsp_servers/solargraph" GEM_PATH="$HOME/.local/share/nvim/lsp_servers/solargraph" bundle install

In your application directory, and it will make sure those application specific gems are in the place there lsp installer puts solargraph.

Could we get this documented in some way? Or does anyone have any ideas to get around having to do this?

williamboman commented 2 years ago

Yeah this will have to be fixed. I don't do Ruby dev so I'm not entirely sure what the best solution would be. One easy thing to fix is to not completely overwrite GEM_PATH, and instead only extend it. This might even fix it altogether?

Slotos commented 2 years ago

The simplest approach would be to simply use the system's way of working with gems, and let the user deal with it. coc-solargraph does exactly that and simplicity never bothered me.

I personally would be content with the following behaviour:

LSP Installer UI allows me to mark servers as "install/update on filetype encounter", which could be two separate flags
When filetype is encountered and the binary for the server is not found, installer offers to install the server or lets me provide the handler of my design
(optional) When starting the LS, and configured to do so, installer offers to update the server if update is available or lets me provide the handler of my design

The idea is to deliberately keep things stupid simple, so that If anything goes wrong, I'd drop down to the shell and deal with gems troubleshooting there, knowing that my editor would follow suit.

williamboman commented 2 years ago

I'm struggling a bit to repro this. Does anyone have a step by step description on how to reproduce this (a dummy repo would be really nice)?

steveclarke commented 2 years ago

I just configured Solargraph on VS Code and am trying to use my newly gained knowledge from getting it working in that editor to make it work in my NeoVim setup. Solargraph usually needs to run using bundle exec within your project directory so it properly detects your .solargraph.yml and .rubocop.yml files to load up the rules specific to your project. bundle exec solargraph. In VS Code I set solargraph.useBundler to true.

I'm very new to NeoVim and LSP so I'm struggling to understand how the language server is configured. i.e. is there an equivalent setting to solargraph.useBundler that I can use here?

I would be happy to create a dummy repo that replicates my working Solargraph setup in a simple Rails app. Would that help any?

Slotos commented 2 years ago

Fresh rails project is sufficient for reproduction

gem install rails
rails new lsp-install-ruby-187
cd repro
nvim config/application.rb

This is my lsp-installer config

    vim.lsp.set_log_level("debug")
    local lsp_installer = require("nvim-lsp-installer")

    -- Register a handler that will be called for all installed servers.
    -- Alternatively, you may also register handlers on specific server instances instead (see example below).
    lsp_installer.on_server_ready(function(server)
      local opts = {
        on_attach = on_attach,
        capabilities = capabilities,
      }

      if server.name == "solargraph" then
        opts.filetypes = { "ruby" }
        opts.flags = { debounce_text_changes = 150, }
        opts.settings = {
          solargraph = {
            diagnostics = true,
            formatting = true,
            }
          }
      end

      server:setup(opts)
    end)

Tailing ~/.cache/nvim/lsp.log, exposes warns akin to:

[ERROR][2022-04-18 01:17:44] .../vim/lsp/rpc.lua:420    "rpc"   "solargraph"    "stderr"    "[WARN] .local/share/nvim/lsp_servers/solargraph/gems/bundler-2.3.11/lib/bundler/definition.rb:481:in `materialize': Could not find rails-7.0.2.3, sprockets-rails-3.4.2, sqlite3-1.4.2, puma-5.6.4, importmap-rails-1.0.3, turbo-rails-1.0.1, stimulus-rails-1.0.4, jbuilder-2.11.5, redis-4.6.0, bootsnap-1.11.1, actioncable-7.0.2.3, actionmailbox-7.0.2.3, actionmailer-7.0.2.3, actionpack-7.0.2.3, actiontext-7.0.2.3, actionview-7.0.2.3, activejob-7.0.2.3, activemodel-7.0.2.3, activerecord-7.0.2.3, activestorage-7.0.2.3, activesupport-7.0.2.3, railties-7.0.2.3, sprockets-4.0.3, nio4r-2.5.8, msgpack-1.5.1, websocket-driver-0.7.5, mail-2.7.1, net-imap-0.2.3, net-pop-0.1.1, net-smtp-0.3.1, rails-dom-testing-2.0.3, rack-2.2.3, rack-test-1.1.0, rails-html-sanitizer-1.4.2, globalid-1.0.0, builder-3.2.4, erubi-1.10.0, marcel-1.0.2, mini_mime-1.1.2, concurrent-ruby-1.1.10, i18n-1.10.0, minitest-5.15.0, tzinfo-2.0.4, method_source-1.0.0, rake-13.0.6, zeitwerk-2.5.4, websocket-extensions-0.1.5, net-protocol-0.1.3, loofah-2.16.0, crass-1.0.6 in any of the sources (Bundler::GemNotFound)...
...
[ERROR][2022-04-18 01:05:37] .../vim/lsp/rpc.lua:420    "rpc"   "solargraph"    "stderr"    "[WARN] Failed to load gems from bundle at...

Running GEM_HOME="$HOME/.local/share/nvim/lsp_servers/solargraph" GEM_PATH="$HOME/.local/share/nvim/lsp_servers/solargraph" bundle install eliminates these errors.

Aside from this, a case for solargraph plugins can be explored by creating .solargraph.yml in aforementioned project's root:

---
include:
- "**/*.rb"
exclude:
- spec/**/*
- test/**/*
- vendor/**/*
- ".bundle/**/*"
require: []
domains: []
reporters:
- rubocop
- require_not_found
formatter:
  rubocop:
    cops: safe
    except: []
    only: []
    extra_args: []
require_paths: []
plugins:
  - solargraph-rails
max_files: 0

Restarting solargraph with this configuration will yield the following message:

[ERROR][2022-04-18 01:08:09] .../vim/lsp/rpc.lua:420    "rpc"   "solargraph"    "stderr"    "[WARN] Failed to load plugin 'solargraph-rails'\n"

One needs to run GEM_HOME="$HOME/.local/share/nvim/lsp_servers/solargraph" GEM_PATH="$HOME/.local/share/nvim/lsp_servers/solargraph" gem install solargraph-rails to install the plugin.

If lsp-installer avoids sandboxing, all these issues are automagically resolved. Switching environments - a common occurrence - will require a fresh solargraph install, which is why an install-on-encounter feature would be a useful QoL addition.

steveclarke commented 2 years ago

If there was a use_bundler option, wouldn't it avoid the necessity to even install a Solargraph server? All that would be required would be to run bundle exec solargraph stdio to launch your project's server.

Slotos commented 2 years ago

Solargraph is not guaranteed to be checked into a Gemfile. It's designed not to require it.

Granted, it's not a bad idea to have try_bundle option to try running it via bundle exec with a fallback to a regular routine. Status code 127 is a reliable indicator - https://github.com/rubygems/rubygems/blob/master/bundler/spec/commands/exec_spec.rb#L376

qbantek commented 2 years ago

Option to run solargraph via bundle exec would be awesome! I would make it the default behavior: try bundler first, if gem not found -> continue as usual.

michaelfranzl commented 2 years ago

For those applications having Solargraph already integrated using Bundler (i.e. in the Gemfile), please consider adding a fallback. In those cases, there would be no need to install anything, but to simply run bundle exec solargraph stdio in an unmodified environment (i.e. no GEM_HOME or GEM_PATH environment variables). In case Rubocop is also in the Gemfile, this would have the advantage that Solargraph would also report linting issues using the applications' own Rubocop configuration.

skcc321 commented 1 year ago

Hi, there I faced the same issue due to inherited_gem in .rubocop.yml. according to the documentation https://docs.rubocop.org/rubocop/configuration.html#inheriting-configuration-from-a-dependency-gem it should be executed via bundle install. As we don't have useBundler option the only way is to run bundle exec solargraph stdio according to the documentation https://github.com/castwide/solargraph#solargraph-and-bundler. The workaround (I would say pretty reliable solution) is found here https://github.com/neovim/nvim-lspconfig/issues/1886#issuecomment-1201356197. The only correction I did is due to the slow bundle info command:

-- from
vim.fn.jobstart("bundle info solargraph", { on_exit
-- to
vim.fn.jobstart("cat Gemfile | grep solargraph", { on_exit = ...

so completely working configuration for lunar_vim looks like this: somewhere in ~/.config/lvim/config.lua looks add next config

-- bundle exec solargraph stdio if Gemfile contains solargraph
vim.list_extend(lvim.lsp.automatic_configuration.skipped_servers, { "solargraph" })

local solargraph_cmd = function()
  local ret_code = nil
  local jid = vim.fn.jobstart("cat Gemfile | grep solargraph", { on_exit = function(_, data) ret_code = data end })
  vim.fn.jobwait({ jid }, 5000)
  if ret_code == 0 then
    return { "bundle", "exec", "solargraph", "stdio" }
  end
  return { "solargraph", "stdio" }
end

local opts = {
  cmd = solargraph_cmd()
}

require("lvim.lsp.manager").setup("solargraph", opts)

AlanWarren commented 1 year ago

I've tried Slotos's suggestion, but I'm still unable to get solargraph to load the solargraph-rails plugin. I've confirmed the solargraph-rails gem is physically located in the same GEM_PATH as the nvim lsp_server's solargraph gem. At this point I'm not sure what else to try.

Does anyone have any other ideas?

Slotos commented 1 year ago

@AlanWarren solargraph-rails is too restrictive in its solargraph version dependency - https://github.com/iftheshoefritz/solargraph-rails/pull/41. It does seem that there was a reason for that, but that's fixed. Find the gem files and apply the change to it, you'll see it load.

Aside from that, switch to mason.nvim and macon-lspconfig.nvim. The latter adds mason installation to gem environment variables instead of replacing them.

AlanWarren commented 1 year ago

fantastic! Thanks so much for sharing.

williamboman / nvim-lsp-installer

GemNotFoundException when using solargraph as a Ruby LSP #187

My use case

Ruby versions