search

wbthomason / packer.nvim

A use-package inspired plugin manager for Neovim. Uses native packages, supports Luarocks dependencies, written in Lua, allows for expressive config
MIT License
7.82k stars 266 forks source link

Modules resolve to different tables addresses when lazyloaded #469

Closed rahulaggarwal965 closed 3 years ago

rahulaggarwal965 commented 3 years ago

Steps to reproduce

lua << EOF
require('packer').startup(function(use)
  use { "mfussenegger/nvim-dap",
    setup = function()
      -- Any mapping that calls require('dap')
      vim.api.nvim_set_keymap('n', '<leader>dd', '<cmd>lua require("dap").continue()<CR>', { noremap = true})
    end,
    config = function()
      -- we print the table address here
      print(require('dap'))
      -- we also set some basic configuration for nvim-dap
      require('dap').adapters.lldb = {
        type = "executable",
        command = "/usr/bin/lldb-vscode",
        name = "lldb"
      }
    end,
    module = "dap"
  }
end
)
EOF
  1. nvim -u init.vim
  2. PackerUpdate
  3. PackerCompile
  4. Open any file
  5. Type dd or call :lua require('dap')
  6. Type :lua print(require'dap')

Actual behaviour

Calling :messages shows that the reqire('dap') table that was loaded in the config function is different from the one that is loaded within the buffer. Example output:

table: 0x41b43308 -- table loaded in the config function
table: 0x41e9a8a0 -- table loaded within the buffer

Additionally we can call vim.inspect on both these tables which shows that they are indeed completely separate modules.

Expected behaviour

I expect the tables to be the same (have the same address) so that anything setup in the config function is carried into the buffer. For some reason the module gets reloaded even though within the config function package.loaded.dap is set to a value. Not sure if this should be the case or if I'm doing something wrong.

packer files

Plugin specification file(s) These are considerably more verbose than the reproducible example. ```lua local execute = vim.api.nvim_command local fn = vim.fn local install_path = fn.stdpath('data')..'/site/pack/packer/start/packer.nvim' if fn.empty(fn.glob(install_path)) > 0 then fn.system({'git', 'clone', 'https://github.com/wbthomason/packer.nvim', install_path}) execute 'packadd packer.nvim' end local packer = require('packer') vim.cmd 'autocmd BufWritePost plugins.lua PackerCompile' return packer.startup(function(use) use "wbthomason/packer.nvim" use "tjdevries/astronauta.nvim" use { "neovim/nvim-lspconfig", module = "lspconfig" } use { "ray-x/lsp_signature.nvim", module = "lsp_signature" } use { "hrsh7th/nvim-compe", config = function() require("plugins.compe") end, event = "InsertEnter" } use { "L3MON4D3/LuaSnip", module = "luasnip" } use { "windwp/nvim-autopairs", after = "nvim-compe", config = function() require("nvim-autopairs").setup() require("nvim-autopairs.completion.compe").setup({ map_cr = true, map_complete = true }) end } use { "lewis6991/gitsigns.nvim", requires = { "nvim-lua/plenary.nvim" }, config = function() require("plugins.gitsigns") end, event = "BufRead" } use { "nvim-telescope/telescope.nvim", requires = { { "nvim-lua/popup.nvim" }, { "nvim-lua/plenary.nvim" }, { "nvim-telescope/telescope-fzy-native.nvim" } }, setup = function() require("plugins.telescope").mappings() end, config = function() require("plugins.telescope").config() end, module = "telescope.builtin" } use { "terrortylor/nvim-comment", config = function() require("nvim_comment").setup() end, event = "BufWinEnter" } use { "mfussenegger/nvim-dap", setup = function() -- Any mapping that calls require('dap') vim.api.nvim_set_keymap('n', 'dd', 'lua require("dap").continue()', { noremap = true}) end, config = function() -- put general dap config here, example: print(require('dap')) require('dap').adapters.lldb = { type = "executable", command = "/usr/bin/lldb-vscode", name = "lldb" } end, module = "dap" } -- use { "rcarriga/nvim-dap-ui", -- after = "nvim-dap", -- } -- TODO(rahul): make this based on filetype use { "nvim-treesitter/nvim-treesitter", config = function() require("plugins.treesitter") end, run = ":TSUpdate" } use { "kyazdani42/nvim-tree.lua", commit = "fd7f60e242205ea9efc9649101c81a07d5f458bb", requires = { "kyazdani42/nvim-web-devicons" }, setup = function() require("plugins.tree").mappings() end, config = function() require("plugins.tree").config() end, cmd = "NvimTreeToggle" } use "christoomey/vim-tmux-navigator" use "RyanMillerC/better-vim-tmux-resizer" use "ChristianChiarulli/nvcode-color-schemes.vim" end ) ```
packer log file Post the contents of ~/.cache/nvim/packer.nvim.log here
packer compiled file Post the contents of `packer_compiled.vim` here ```lua -- Automatically generated packer.nvim plugin loader code if vim.api.nvim_call_function('has', {'nvim-0.5'}) ~= 1 then vim.api.nvim_command('echohl WarningMsg | echom "Invalid Neovim version for packer.nvim! | echohl None"') return end vim.api.nvim_command('packadd packer.nvim') local no_errors, error_msg = pcall(function() local time local profile_info local should_profile = false if should_profile then local hrtime = vim.loop.hrtime profile_info = {} time = function(chunk, start) if start then profile_info[chunk] = hrtime() else profile_info[chunk] = (hrtime() - profile_info[chunk]) / 1e6 end end else time = function(chunk, start) end end local function save_profiles(threshold) local sorted_times = {} for chunk_name, time_taken in pairs(profile_info) do sorted_times[#sorted_times + 1] = {chunk_name, time_taken} end table.sort(sorted_times, function(a, b) return a[2] > b[2] end) local results = {} for i, elem in ipairs(sorted_times) do if not threshold or threshold and elem[2] > threshold then results[i] = elem[1] .. ' took ' .. elem[2] .. 'ms' end end _G._packer = _G._packer or {} _G._packer.profile_output = results end time([[Luarocks path setup]], true) local package_path_str = "/home/infinity/.cache/nvim/packer_hererocks/2.0.5/share/lua/5.1/?.lua;/home/infinity/.cache/nvim/packer_hererocks/2.0.5/share/lua/5.1/?/init.lua;/home/infinity/.cache/nvim/packer_hererocks/2.0.5/lib/luarocks/rocks-5.1/?.lua;/home/infinity/.cache/nvim/packer_hererocks/2.0.5/lib/luarocks/rocks-5.1/?/init.lua" local install_cpath_pattern = "/home/infinity/.cache/nvim/packer_hererocks/2.0.5/lib/lua/5.1/?.so" if not string.find(package.path, package_path_str, 1, true) then package.path = package.path .. ';' .. package_path_str end if not string.find(package.cpath, install_cpath_pattern, 1, true) then package.cpath = package.cpath .. ';' .. install_cpath_pattern end time([[Luarocks path setup]], false) time([[try_loadstring definition]], true) local function try_loadstring(s, component, name) local success, result = pcall(loadstring(s)) if not success then vim.schedule(function() vim.api.nvim_notify('packer.nvim: Error running ' .. component .. ' for ' .. name .. ': ' .. result, vim.log.levels.ERROR, {}) end) end return result end time([[try_loadstring definition]], false) time([[Defining packer_plugins]], true) _G.packer_plugins = { LuaSnip = { loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/LuaSnip" }, ["astronauta.nvim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/astronauta.nvim" }, ["better-vim-tmux-resizer"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/better-vim-tmux-resizer" }, ["gitsigns.nvim"] = { config = { "\27LJ\1\0020\0\0\2\0\2\0\0044\0\0\0%\1\1\0>\0\2\1G\0\1\0\21plugins.gitsigns\frequire\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/gitsigns.nvim" }, ["lsp_signature.nvim"] = { loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/lsp_signature.nvim" }, ["nvcode-color-schemes.vim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/nvcode-color-schemes.vim" }, ["nvim-autopairs"] = { config = { "\27LJ\1\2\1\0\0\2\0\5\0\f4\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\0014\0\0\0%\1\3\0>\0\2\0027\0\2\0003\1\4\0>\0\2\1G\0\1\0\1\0\2\17map_complete\2\vmap_cr\2$nvim-autopairs.completion.compe\nsetup\19nvim-autopairs\frequire\0" }, load_after = { ["nvim-compe"] = true }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-autopairs" }, ["nvim-comment"] = { config = { "\27LJ\1\2:\0\0\2\0\3\0\0064\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\1G\0\1\0\nsetup\17nvim_comment\frequire\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-comment" }, ["nvim-compe"] = { after = { "nvim-autopairs" }, after_files = { "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-compe/after/plugin/compe.vim" }, config = { "\27LJ\1\2-\0\0\2\0\2\0\0044\0\0\0%\1\1\0>\0\2\1G\0\1\0\18plugins.compe\frequire\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-compe" }, ["nvim-dap"] = { config = { "\27LJ\1\2‘\1\0\0\3\0\6\0\f4\0\0\0004\1\1\0%\2\2\0>\1\2\0=\0\0\0014\0\1\0%\1\2\0>\0\2\0027\0\3\0003\1\5\0:\1\4\0G\0\1\0\1\0\3\tname\tlldb\fcommand\25/usr/bin/lldb-vscode\ttype\15executable\tlldb\radapters\bdap\frequire\nprint\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-dap" }, ["nvim-lspconfig"] = { loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-lspconfig" }, ["nvim-tree.lua"] = { commands = { "NvimTreeToggle" }, config = { "\27LJ\1\2;\0\0\2\0\3\0\0064\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\1G\0\1\0\vconfig\17plugins.tree\frequire\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/nvim-tree.lua" }, ["nvim-treesitter"] = { config = { "\27LJ\1\0022\0\0\2\0\2\0\0044\0\0\0%\1\1\0>\0\2\1G\0\1\0\23plugins.treesitter\frequire\0" }, loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/nvim-treesitter" }, ["nvim-web-devicons"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/nvim-web-devicons" }, ["packer.nvim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/packer.nvim" }, ["plenary.nvim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/plenary.nvim" }, ["popup.nvim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/popup.nvim" }, ["telescope-fzy-native.nvim"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/telescope-fzy-native.nvim" }, ["telescope.nvim"] = { config = { "\27LJ\1\2@\0\0\2\0\3\0\0064\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\1G\0\1\0\vconfig\22plugins.telescope\frequire\0" }, loaded = false, needs_bufread = false, path = "/home/infinity/.local/share/nvim/site/pack/packer/opt/telescope.nvim" }, ["vim-tmux-navigator"] = { loaded = true, path = "/home/infinity/.local/share/nvim/site/pack/packer/start/vim-tmux-navigator" } } time([[Defining packer_plugins]], false) local module_lazy_loads = { ["^dap"] = "nvim-dap", ["^lsp_signature"] = "lsp_signature.nvim", ["^lspconfig"] = "nvim-lspconfig", ["^luasnip"] = "LuaSnip", ["^telescope%.builtin"] = "telescope.nvim" } local lazy_load_called = {['packer.load'] = true} local function lazy_load_module(module_name) local to_load = {} if lazy_load_called[module_name] then return nil end lazy_load_called[module_name] = true for module_pat, plugin_name in pairs(module_lazy_loads) do if not _G.packer_plugins[plugin_name].loaded and string.match(module_name, module_pat)then to_load[#to_load + 1] = plugin_name end end require('packer.load')(to_load, {module = module_name}, _G.packer_plugins) end if not vim.g.packer_custom_loader_enabled then table.insert(package.loaders, 1, lazy_load_module) vim.g.packer_custom_loader_enabled = true end -- Setup for: telescope.nvim time([[Setup for telescope.nvim]], true) try_loadstring("\27LJ\1\2B\0\0\2\0\3\0\0064\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\1G\0\1\0\rmappings\22plugins.telescope\frequire\0", "setup", "telescope.nvim") time([[Setup for telescope.nvim]], false) -- Setup for: nvim-dap time([[Setup for nvim-dap]], true) try_loadstring('\27LJ\1\2ƒ\1\0\0\5\0\a\0\t4\0\0\0007\0\1\0007\0\2\0%\1\3\0%\2\4\0%\3\5\0003\4\6\0>\0\5\1G\0\1\0\1\0\1\fnoremap\2+lua require("dap").continue()\15dd\6n\20nvim_set_keymap\bapi\bvim\0', "setup", "nvim-dap") time([[Setup for nvim-dap]], false) -- Setup for: nvim-tree.lua time([[Setup for nvim-tree.lua]], true) try_loadstring("\27LJ\1\2=\0\0\2\0\3\0\0064\0\0\0%\1\1\0>\0\2\0027\0\2\0>\0\1\1G\0\1\0\rmappings\17plugins.tree\frequire\0", "setup", "nvim-tree.lua") time([[Setup for nvim-tree.lua]], false) -- Config for: nvim-treesitter time([[Config for nvim-treesitter]], true) try_loadstring("\27LJ\1\0022\0\0\2\0\2\0\0044\0\0\0%\1\1\0>\0\2\1G\0\1\0\23plugins.treesitter\frequire\0", "config", "nvim-treesitter") time([[Config for nvim-treesitter]], false) -- Command lazy-loads time([[Defining lazy-load commands]], true) vim.cmd [[command! -nargs=* -range -bang -complete=file NvimTreeToggle lua require("packer.load")({'nvim-tree.lua'}, { cmd = "NvimTreeToggle", l1 = , l2 = , bang = , args = }, _G.packer_plugins)]] time([[Defining lazy-load commands]], false) vim.cmd [[augroup packer_load_aucmds]] vim.cmd [[au!]] -- Event lazy-loads time([[Defining lazy-load event autocommands]], true) vim.cmd [[au BufWinEnter * ++once lua require("packer.load")({'nvim-comment'}, { event = "BufWinEnter *" }, _G.packer_plugins)]] vim.cmd [[au InsertEnter * ++once lua require("packer.load")({'nvim-compe'}, { event = "InsertEnter *" }, _G.packer_plugins)]] vim.cmd [[au BufRead * ++once lua require("packer.load")({'gitsigns.nvim'}, { event = "BufRead *" }, _G.packer_plugins)]] time([[Defining lazy-load event autocommands]], false) vim.cmd("augroup END") if should_profile then save_profiles() end end) if not no_errors then vim.api.nvim_command('echohl ErrorMsg | echom "Error in packer_compiled: '..error_msg..'" | echom "Please check your config for correctness" | echohl None') end ```
wbthomason commented 3 years ago

Fascinating issue, thanks for your report! I think this is a problem specifically with the module lazy-loader key. Do you see the same unexpected behavior if you use a different lazy-loader, or no lazy-loader at all?

rahulaggarwal965 commented 3 years ago

So I have a couple of plugins that employ the module feature. However, the way I call require on them does different things.

Lazy Loading lspconfig and lsp_signature with the module key
I lazy load `lspconfig` by calling `require("lspconfig")` in ftplugin files ```lua -- nvim/ftplugin/*.lua -- require("lspconfig").*.setup { -- SNIP -- on_attach = function(client) require("lsp").on_attach(client) end, -- SNIP -- } ``` Additionally, I also lazyload `lsp_signature` in `require("lsp").on_attach` here: ```lua -- nvim/lua/lsp.lua -- local M = {} M.on_attach = function(client) require('lsp_signature').on_attach({ bind = true, hint_enable = true, fix_pos = false, hint_prefix = " ", hint_scheme = "String", handler_opts = { border = "single" } }) -- SNIP -- end ``` These mappings work perfectly fine using the `module` option with lspconfig and lsp_signature. However, they are not being required in a config function but rather in a ftplugin sense, which might have different mechanics.
Lazy Loading luasnip
I do load `LuaSnip` within the `config` function for `nvim-compe` which can be seen here ```lua require('packer').startup(function(use) use { "hrsh7th/nvim-compe", config = function() require("plugins.compe") end, event = "InsertEnter" } use { "L3MON4D3/LuaSnip", module = "luasnip" } end ``` the `require('plugins.compe')` has a call to `require('luasnip")` but it DOES NOT actually use that call in the sense that we don't access or change any of luasnips table members. Instead, we just map things like this: ```lua -- nvim/lua/plugins/compe.lua -- local luasnip = require("luasnip") _G.tab_complete = function() if vim.fn.pumvisible() == 1 then return t "" elseif luasnip.expand_or_jumpable() then return t "luasnip-expand-or-jump" else return t "" end end _G.s_tab_complete = function() if vim.fn.pumvisible() == 1 then return t "" elseif luasnip.jumpable(-1) then return t "luasnip-jump-prev" else return t "" end end ``` So in that sense, it makes sense that the plugin still works. I suspect that changes on the specific module key don't propagate from within config functions.
The most curious case is with lazyloading telescope through the module feature
The telescope config looks like this: ```lua require('packer').startup(function(use) use { "nvim-telescope/telescope.nvim", requires = { { "nvim-lua/popup.nvim" }, { "nvim-lua/plenary.nvim" }, { "nvim-telescope/telescope-fzy-native.nvim" } }, setup = function() require("plugins.telescope").mappings() end, config = function() require("plugins.telescope").config() end, module = "telescope.builtin" } end ``` The `require('plugins.telescope')` calls this code: ```lua -- nvim/lua/plugins/telescope.lua -- local M = {} M.mappings = function() vim.api.nvim_set_keymap( 'n', 'f', 'lua require("telescope.builtin").find_files()', { noremap = true }) -- SNIP -- end M.config = function() print("telescope address: ", require'telescope') print("telescope.builtin address: ", require'telescope.builtin') require('telescope').setup{ -- USER CONFIG -- } end ``` So we use the module key `telescope.builtin` and then in the setup function define mappings that will require that module key. Once that happens, the telescope config will be loaded and M.config will be run. The interesting thing is that the table address for `require('telescope.builtin')` is different within M.config and the actual general file, but `require('telescope')` is the same table address. We can see this by opening a new buffer and entering the following commands ```lua print("buffer telescope address: ", require'telescope') print("buffer telescope.builtin address: ", require'telescope.builtin') ``` The output is ```lua telescope address: table: 0x4154df40 telescope.builtin address: table: 0x4154fa08 buffer telescope address: table: 0x4154df40 buffer telescope.builtin address: table: 0x4154d6e8 ``` The telescope addresses are the same, but the `telescope.builtin` address which is the module key, is not. So it seems to me that the specific module key that is passed in is somehow being reloaded after the config function. There is definitely a bug in the interplay between the module key and the config function, but a different require like `telescope` itself resolves normally in the config function and a general buffer because it is NOT the module key.

I realize that you might need a bit more context so if you do, I included a link to github with the full directory structure, etc: Full Config

rahulaggarwal965 commented 3 years ago

I also just tried removing the module key from nvim-dap and having it load at start and the issues went away, so it definitely has something to do with the interplay between the config key and module key. The new test was just:

require('packer').startup(function(use)
    use { "mfussenegger/nvim-dap",
        setup = function()
            require("plugins.dap").mappings()
        end,
        config = function()
            require("plugins.dap").config()
        end
    }
end

Both tables were the same:

config table: 0x4173fd30
buffer table: 0x4173fd30
wbthomason commented 3 years ago

Just to confirm, you are seeing that config values are not persisting, right? If using the result of require is working as expected (i.e. config values are persisting between requires), then I'm not very concerned that different tables get returned here. If things are breaking, though, then we need to understand why the module loader causes this.

wbthomason commented 3 years ago

Hmm. If we require dap multiple times within the config function, it is the same table. It's only when we require it afterward that we get a different table. Also, I confirmed that config values are not persisting.

wbthomason commented 3 years ago

Thanks, this was the weirdest and most fun bug I've had in a while! Your intuition about what was going on was pretty close - I've explained the problem (and solution) in #472. If you get a chance, @rahulaggarwal965, could you please confirm that #472 fixes this for you without breaking anything else, and I'll go ahead and merge it?

rahulaggarwal965 commented 3 years ago

I checked out the branch and I think the fix did stop the rerunning of modules. However, if we call require'dap' in the config function, (like in the repro example), then when we call print(require'dap') from a general buffer we now get true, which is also a bug. This causes errors when we call mappings that require dap as they now try to index from a table that is actually the value true.

input:

print(require'dap')

output:

table: 0x41b4cca8 -- from print(require'dap') within the config() function of use { dap }
true -- from the print within the buffer

I tried digging around in the code

-- SNIP --
  local log = require('packer.log')
  if #to_load > 0 then
    log.info(vim.inspect(to_load) .. " " .. module_name) -- logs: {'nvim-dap'} dap
    require('packer.load')(to_load, {module = module_name}, _G.packer_plugins)
    log.info(vim.inspect(package.loaded[module_name])) -- logs: { --- Full Dap Table -- }
    if package.loaded[module_name] then
      return function(modname) 
          print(package.loaded[modname]) -- userdata: 0x7f9b7857cea4
          print(vim.inspect(package.loaded[modname])) -- <userdata 1>
          print(vim.inspect(getmetatable(package.loaded[modname]))) -- nil
          return package.loaded[modname] -- this somehow gets resolved to true on the caller side
      end
    end
-- SNIP --

Clearly, within the package.loaders function call, package.loaded[modname] is acting weird. To solve this, I changed this to

local mod = package.loaded[module_name] -- cache module
if mod then
  return function(modname)
    return mod
  end
end

I'm not sure if the modname parameter is necessary or if this is the best way to solve this, but at least it's a start.

wbthomason commented 3 years ago

Interesting - I thought I had fixed that behavior, which I did encounter while making the original PR. Your workaround should probably be fine for this.