Lazy require language server configurations to avoid increasing neovim startup time

[ms-jpq]

Hi there,

I noticed that my vim startup time doubled, when I added in a bunch of lsp configs (I added many).

So I lazy loaded all of mine setup(...) in FileType autocmd.

I had to dig around a bit and add in lsp.<client>.manager.try_add() for each client, or else the first buffer of FileType would not be loaded.

Of course this isn't too difficult, but I think there are some room for improvement.

We could perhaps add in a param for require("nvim_lsp").<client>.setup(...) such that it does this automatically.

This would not be a breaking change by any means, and should speed up things for our users.

Edit:

I don't know why this is labled as a bug automatically, how do I change it?

[tjdevries]

Can you show what your final config looked like?

[ms-jpq]

Sure thing it goes something like this:

bash.lua

local ftp = require "libs/ftp"
local ft = {"python"}

local lsp = function ()
  if not bindings.executable("pyls") then
    return
  end
  local lsp = require "nvim_lsp"
  lsp.pyls.setup{}
  lsp.pyls.manager.try_add()
end
ftp.defer(ft, lsp)

libs/ftp.lua

local set = require "libs/set"
local registry = require "libs/registry"

local _ftp = set.new()

local defer = function (ft, ftplugin)
  set.add(_ftp, ftplugin)
  local ftp = function (kill)
    if not set.contains(_ftp, ftplugin) then
      return
    end
    set.subtract(_ftp, ftplugin)
    ftplugin()
    kill()
    print("-- LSP 加载: " .. table.concat(ft, ",") .. " --")
  end
  registry.auto("FileType", ftp, ft)
end

libs/registry.lua

local _callbacks = {}
local inc = std.count()

local auto = function (events, func, filter, modifiers)

  local evnts = std.wrap(events)
  local events = table.concat(evnts, ",")
  local filter = table.concat(std.wrap(filter or "*"), ",")
  local modifiers = " " .. table.concat(std.wrap(modifiers or {}), " ")
  local idx = inc()
  local group = "augroup " .. idx
  local cls = "autocmd!"
  local cmd = "autocmd " .. events .. " " .. filter .. modifiers .. " lua require('" .. _registry .. "').call(" .. idx .. ")"
  local done = "augroup END"

  for event in ipairs(evnts) do
    assert(fn.exists("##" .. event))
  end

  _callbacks[idx] = func
  api.nvim_command(group)
  api.nvim_command(cls)
  api.nvim_command(cmd)
  api.nvim_command(done)

  return function ()
    remove(idx)
  end
end

The way my config is, the code posted is way longer than necessary for doing just lazy loading, we can easily simplify this, but you get the gitst of things.

[kosayoda]

@ms-jpq hi, I was looking to emulate your workaround, and I am having a bit of trouble.

libs/ftp.lua

local set = require "libs/set"

I'm assuming you have a set implementation in this file?

libs/registry.lua

local inc = std.count()

May I know what this std is?

Thanks!

[ms-jpq]

@kosayoda

I wrote an stdlib here

[mjlbach]

Just wanted to throw some numbers in here:

With every language server installed

event                  time percent plot
sourcing vimrc file(  95.00   24.02 ██████████████████████████
filetype.vim          47.38   11.98 █████████████
onedark.vim           43.23   10.93 ███████████▉
polyglot.vim          32.86    8.31 █████████

With none

event                  time percent plot
filetype.vim          46.14   13.87 ██████████████████████████
sourcing vimrc file(  45.90   13.80 █████████████████████████▉
onedark.vim           42.37   12.74 ███████████████████████▉
polyglot.vim          30.13    9.06 █████████████████

It seems like the majority of this time is spent on importing the module, not on the setup:

Time required to require each language server

``` 0 0.000716207 0.000716207 als 1 0.000947072 0.001663279 angularls 2 0.000697 0.002360279 bashls 3 0.000706567 0.003066846 ccls 4 0.000814418 0.003881264 clangd 5 0.000754559 0.004635823 clojure_lsp 6 0.000769007 0.00540483 cmake 7 0.000688428 0.006093258 codeqlls 8 0.00069076 0.006784018 cssls 9 0.000947254 0.007731272 dartls 10 0.000682198 0.00841347 dockerls 11 0.00082909 0.00924256 efm 12 0.000817084 0.010059644 elixirls 13 0.000743021 0.010802665 elmls 14 0.000678039 0.011480704 flow 15 0.00064037 0.012121074 fortls 16 0.000632705 0.012753779 gdscript 17 0.00074599 0.013499769 ghcide 18 0.000679864 0.014179633 gopls 19 0.000652367 0.014832 groovyls 20 0.000620872 0.015452872 hie 21 0.000809251 0.016262123 hls 22 0.000717957 0.01698008 html 23 0.000679174 0.017659254 intelephense 24 0.000906135 0.018565389 jdtls 25 0.000826396 0.019391785 jedi_language_server 26 0.000745125 0.02013691 jsonls 27 0.000673965 0.020810875 julials 28 0.000783778 0.021594653 kotlin_language_server 29 0.000747691 0.022342344 leanls 30 0.00067887 0.023021214 metals 31 0.000663578 0.023684792 nimls 32 0.00071578 0.024400572 ocamlls 33 0.000649891 0.025050463 ocamllsp 34 0.000722818 0.025773281 omnisharp 35 0.000693278 0.026466559 perlls 36 0.000682847 0.027149406 purescriptls 37 0.000636292 0.027785698 pyls 38 0.000682608 0.028468306 pyls_ms 39 0.000681713 0.029150019 pyright 40 0.000712235 0.029862254 r_language_server 41 0.000694758 0.030557012 rls 42 0.000626499 0.031183511 rnix 43 0.000607595 0.031791106 rome ```

You can try out this branch, which basically has everything disabled except for the bare imports, and the startup time is comparable to the main branch. Profiling including in the on_setup function and before and after requiring each LS module.

https://github.com/mjlbach/nvim-lspconfig/tree/profiling

[mjlbach]

local ftp = require "libs/ftp"
local ft = {"python"}

local lsp = function ()
  if not bindings.executable("pyls") then
    return
  end
  local lsp = require "nvim_lsp"
  lsp.pyls.setup{}
end
ftp.defer(ft, lsp)

We could implement something like this (really, it's deferring the 'require("lspconfig/pyright")' in the call to the table in lsp.pyls that saves the time), but then the issue is we'd need to keep a separate table mapping the filetype to each server's lua module so we know how to define the autocommands.

The other option is implementing an async loop to load all of the configs into a table (haven't dug into this too much), and then generate the mapping from filetype to autocommand dynamically, which has the advantage of not requiring either another github action (to autogenerate the filetype table from commands)

[dagadbm]

hey guys I also have noticed a big startup difference ever since i moved from coc to native lsp and i think this is the main reason. I am way out of my depth here but from a "user perspective" and really simplying the problem:

When I do the LspInfo command there is already information there about the type of file that it supports, cant it be done from that information create the auto commands automatically for that and just do the real setup() and launch of the lsp there?

So the user when he would type setup() would basically just store the object in memory and only when a buffer of that filetype opened would it run the "real setup"

I have very little experience on this I just migrated everything from vim to lua and coc to lsp yesterday but I was thinking that this should be implementedd either on one of those lsp-install repositories or in lspconfig.

[mjlbach]

Did you see my profiling data? The overhead to startup is about 1 ms per enabled server.

[mjlbach]

Also the way LspInfo works is orthogonal to this problem (I wrote it), it only tests servers that have already had setup({}). Parsing the 70+ files on startup to figure out which to lazily load is the performance bottleneck. My nvim startups up in 70 ms FWIW, so I would start with our minimal config and diagnose your setup for issues.

[dagadbm]

It’s something i have difficulty knowing how to do.

I also changed to Packer and I feel it’s slower as well.

any tips on how I can profile stuff ? It’s crazy how so many ppl know how to do this so easily and I never find a consistent and easy way to profile.

[akinsho]

Just as a heads up we're currently working on adding built in profiling to packer over in https://github.com/wbthomason/packer.nvim/pull/221 so you should be able to get a clearer picture of what is actually causing your startup time issues.

[weizheheng]

Hi, I hope I understand this issue correctly, the startup time we are referring here is the time until the lsp comes into effect right? Personally, I have 3 language server setup, tsserver, cssls and solargraph. I have no problem with tsserver and cssls, both started almost immediately after I visit related file. The only problem I am having is for Solargraph, it seems like it takes around 6-8 seconds to get initialized. Also, in the profiling above, Solargraph is not included. Appreciate if anyone have any insight on this. Thank you and have a very nice day!

[ INFO ] 2021-04-18T08:58:20+0800 ] ...twares/nvim-osx64/share/nvim/runtime/lua/vim/lsp/rpc.lua:311 ]   "Starting RPC client"   {  args = { "stdio" },  cmd = "solargraph",  extra = {}}
[ DEBUG ] 2021-04-18T08:58:20+0800 ] .../Softwares/nvim-osx64/share/nvim/runtime/lua/vim/lsp.lua:819 ]  "LSP[solargraph]"   "initialize_params" {  capabilities = {    callHierarchy = {      dynamicRegistration = false,      <metatable> = <1>{        __tostring = <function 1>      }    },    textDocument = {      codeAction = {        codeActionLiteralSupport = {          codeActionKind = {            valueSet = { "", "Empty", "QuickFix", "Refactor", "RefactorExtract", "RefactorInline", "RefactorRewrite", "Source", "SourceOrganizeImports", "quickfix", "refactor", "refactor.extract", "refactor.inline", "refactor.rewrite", "source", "source.organizeImports" },            <metatable> = <table 1>          },          <metatable> = <table 1>        },        dynamicRegistration = false,        <metatable> = <table 1>      },      completion = {        completionItem = {          commitCharactersSupport = false,          deprecatedSupport = false,          documentationFormat = { "markdown", "plaintext" },          preselectSupport = false,          snippetSupport = false,          <metatable> = <table 1>        },        completionItemKind = {          valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 },          <metatable> = <table 1>        },        contextSupport = false,        dynamicRegistration = false,        <metatable> = <table 1>      },      declaration = {        linkSupport = true,        <metatable> = <table 1>      },      definition = {        linkSupport = true,        <metatable> = <table 1>      },      documentHighlight = {        dynamicRegistration = false,        <metatable> = <table 1>      },      documentSymbol = {        dynamicRegistration = false,        hierarchicalDocumentSymbolSupport = true,        symbolKind = {          valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 },          <metatable> = <table 1>        },        <metatable> = <table 1>      },      hover = {        contentFormat = { "markdown", "plaintext" },        dynamicRegistration = false,        <metatable> = <table 1>      },      implementation = {        linkSupport = true,        <metatable> = <table 1>      },      publishDiagnostics = {        relatedInformation = true,        tagSupport = {          valueSet = { 1, 2 },          <metatable> = <table 1>        },        <metatable> = <table 1>      },      references = {        dynamicRegistration = false,        <metatable> = <table 1>      },      rename = {        dynamicRegistration = false,        prepareSupport = true,        <metatable> = <table 1>      },      signatureHelp = {        dynamicRegistration = false,        signatureInformation = {          documentationFormat = { "markdown", "plaintext" },          <metatable> = <table 1>        },        <metatable> = <table 1>      },      synchronization = {        didSave = true,        dynamicRegistration = false,        willSave = false,        willSaveWaitUntil = false,        <metatable> = <table 1>      },      typeDefinition = {        linkSupport = true,        <metatable> = <table 1>      },      <metatable> = <table 1>    },    window = {      showDocument = {        support = false,        <metatable> = <table 1>      },      showMessage = {        messageActionItem = {          additionalPropertiesSupport = false,          <metatable> = <table 1>        },        <metatable> = <table 1>      },      workDoneProgress = true,      <metatable> = <table 1>    },    workspace = {      applyEdit = true,      configuration = true,      symbol = {        dynamicRegistration = false,        hierarchicalWorkspaceSymbolSupport = true,        symbolKind = {          valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 },          <metatable> = <table 1>        },        <metatable> = <table 1>      },      workspaceEdit = {        resourceOperations = { "rename", "create", "delete" },        <metatable> = <table 1>      },      workspaceFolders = true,      <metatable> = <table 1>    }  },  clientInfo = {    name = "Neovim",    version = "0.5.0"  },  initializationOptions = vim.empty_dict(),  processId = 54361,  rootPath = "/Users/marcusheng/Workspace/project-tapir",  rootUri = "file:///Users/marcusheng/Workspace/project-tapir",  trace = "off",  workspaceFolders = { {      name = "/Users/marcusheng/Workspace/project-tapir",      uri = "file:///Users/marcusheng/Workspace/project-tapir"    } }}
[ DEBUG ] 2021-04-18T08:58:20+0800 ] ...twares/nvim-osx64/share/nvim/runtime/lua/vim/lsp/rpc.lua:390 ]  "rpc.send.payload"  {  id = 1,  jsonrpc = "2.0",  method = "initialize",  params = {    capabilities = {      callHierarchy = {        dynamicRegistration = false,        <metatable> = <1>{          __tostring = <function 1>        }      },      textDocument = {        codeAction = {          codeActionLiteralSupport = {            codeActionKind = {              valueSet = { "", "Empty", "QuickFix", "Refactor", "RefactorExtract", "RefactorInline", "RefactorRewrite", "Source", "SourceOrganizeImports", "quickfix", "refactor", "refactor.extract", "refactor.inline", "refactor.rewrite", "source", "source.organizeImports" },              <metatable> = <table 1>            },            <metatable> = <table 1>          },          dynamicRegistration = false,          <metatable> = <table 1>        },        completion = {          completionItem = {            commitCharactersSupport = false,            deprecatedSupport = false,            documentationFormat = { "markdown", "plaintext" },            preselectSupport = false,            snippetSupport = false,            <metatable> = <table 1>          },          completionItemKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 },            <metatable> = <table 1>          },          contextSupport = false,          dynamicRegistration = false,          <metatable> = <table 1>        },        declaration = {          linkSupport = true,          <metatable> = <table 1>        },        definition = {          linkSupport = true,          <metatable> = <table 1>        },        documentHighlight = {          dynamicRegistration = false,          <metatable> = <table 1>        },        documentSymbol = {          dynamicRegistration = false,          hierarchicalDocumentSymbolSupport = true,          symbolKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 },            <metatable> = <table 1>          },          <metatable> = <table 1>        },        hover = {          contentFormat = { "markdown", "plaintext" },          dynamicRegistration = false,          <metatable> = <table 1>        },        implementation = {          linkSupport = true,          <metatable> = <table 1>        },        publishDiagnostics = {          relatedInformation = true,          tagSupport = {            valueSet = { 1, 2 },            <metatable> = <table 1>          },          <metatable> = <table 1>        },        references = {          dynamicRegistration = false,          <metatable> = <table 1>        },        rename = {          dynamicRegistration = false,          prepareSupport = true,          <metatable> = <table 1>        },        signatureHelp = {          dynamicRegistration = false,          signatureInformation = {            documentationFormat = { "markdown", "plaintext" },            <metatable> = <table 1>          },          <metatable> = <table 1>        },        synchronization = {          didSave = true,          dynamicRegistration = false,          willSave = false,          willSaveWaitUntil = false,          <metatable> = <table 1>        },        typeDefinition = {          linkSupport = true,          <metatable> = <table 1>        },        <metatable> = <table 1>      },      window = {        showDocument = {          support = false,          <metatable> = <table 1>        },        showMessage = {          messageActionItem = {            additionalPropertiesSupport = false,            <metatable> = <table 1>          },          <metatable> = <table 1>        },        workDoneProgress = true,        <metatable> = <table 1>      },      workspace = {        applyEdit = true,        configuration = true,        symbol = {          dynamicRegistration = false,          hierarchicalWorkspaceSymbolSupport = true,          symbolKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 },            <metatable> = <table 1>          },          <metatable> = <table 1>        },        workspaceEdit = {          resourceOperations = { "rename", "create", "delete" },          <metatable> = <table 1>        },        workspaceFolders = true,        <metatable> = <table 1>      }    },    clientInfo = {      name = "Neovim",      version = "0.5.0"    },    initializationOptions = vim.empty_dict(),    processId = 54361,    rootPath = "/Users/marcusheng/Workspace/project-tapir",    rootUri = "file:///Users/marcusheng/Workspace/project-tapir",    trace = "off",    workspaceFolders = { {        name = "/Users/marcusheng/Workspace/project-tapir",        uri = "file:///Users/marcusheng/Workspace/project-tapir"      } }  }}
[ ERROR ] 2021-04-18T08:58:21+0800 ] ...twares/nvim-osx64/share/nvim/runtime/lua/vim/lsp/rpc.lua:457 ]  "rpc"   "solargraph"    "stderr"    "Solargraph is listening on stdio PID=54495\n"
[ ERROR ] 2021-04-18T08:58:23+0800 ] ...twares/nvim-osx64/share/nvim/runtime/lua/vim/lsp/rpc.lua:457 ]  "rpc"   "solargraph"    "stderr"    '(none):90: warning: key :"" is duplicated and overwritten on line 90\n'
[ ERROR ] 2021-04-18T08:58:28+0800 ] ...twares/nvim-osx64/share/nvim/runtime/lua/vim/lsp/rpc.lua:457 ]  "rpc"   "solargraph"    "stderr"    "[ANY] Solargraph initialized (6.501735999248922 seconds)\n"

[mjlbach]

No, this issue is about the impact to startup time generally even when not opening a file. Requiring a lua file adds about 0.7 ms per file, so if you were to set the configuration for 70+ language servers (invoking Setup({}) doesn't really matter), then you'd add about 60 ms to your startup time. The reason solargraph takes 6-8 seconds is because it's a slow language server, nothing we can do about that here.

[mjlbach]

Made a mistake, please try https://github.com/neovim/nvim-lspconfig/pull/861

[akinsho]

@mjlbach I gave your branch a try having been watching this issue and saw a roughly 3ms drop in startup time from 8ms to 5ms I should say that the test isn't entirely clean since the initialisation of nvim-lspinstall is included in my config so that adds some cost to my numbers but that was the same between runs and the only thing I changed was to your branch.

Hope that's of some value 🤷🏾‍♂️

[mjlbach]

How many language servers do you have installed? I'm wondering if nvim-lspinstall is requiring configs, because I saw a drop of about 40 ms with 15 or so language servers installed.

[akinsho]

About 5 configured atm