PackerInstall crashes badly on nixos

[noib3]

os: NixOS 20.09 (Nightingale) nvim --version: NVIM v0.5.0-dev

I'm trying to use packer on a remote server running nixos, but every time I try to :PackerInstall neovim just crashes.

When I open neovim I get this error message:

Error in packer_compiled: Vim(lua):E5108: Error executing lua /home/nix/.config/nvim/lua/config/colorizer.lua:3: module '
colorizer' not found:^@^Ino field package.preload['colorizer']^@^Ino file './colorizer.lua'^@^Ino file '/nix/store/s2q178
smkfc62l0lg2ir8hp3qqyyc8yl-luajit-2.1.0-2020-12-28/share/luajit-2.1.0-beta3/colorizer.lua'^@^Ino file '/usr/local/share/l
ua/5.1/colorizer.lua'^@^Ino file '/usr/local/share/lua/5.1/colorizer/init.lua'^@^Ino file '/nix/store/s2q178smkfc62l0lg2i
r8hp3qqyyc8yl-luajit-2.1.0-2020-12-28/share/lua/5.1/colorizer.lua'^@^Ino file '/nix/store/s2q178smkfc62l0lg2ir8hp3qqyyc8y
l-luajit-2.1.0-2020-12-28/share/lua/5.1/colorizer/init.lua'^@^Ino file '/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta
3/share/lua/5.1/colorizer.lua'^@^Ino file '/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/share/lua/5.1/colorizer/ini
t.lua'^@^Ino file '/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/lib/luarocks/rocks-5.1/colorizer.lua'^@^Ino file '/
home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/lib/luarocks/rocks-5.1/colorizer/init.lua'^@^Ino file './colorizer.so'^
@^Ino file '/usr/local/lib/lua/5.1/colorizer.so'^@^Ino file '/nix/store/s2q178smkfc62l0lg2ir8hp3qqyyc8yl-luajit-2.1.0-202
0-12-28/lib/lua/5.1/colorizer.so'^@^Ino file '/usr/local/lib/lua/5.1/loadall.so'^@^Ino file '/home/nix/.cache/nvim/packer
_hererocks/2.1.0-beta3/lib/lua/5.1/colorizer.so'
Please check your config for correctness

this is my packer_compiled.vim

" Automatically generated packer.nvim plugin loader code

if !has('nvim-0.5')
  echohl WarningMsg
  echom "Invalid Neovim version for packer.nvim!"
  echohl None
  finish
endif

packadd packer.nvim

try

lua << END
local package_path_str = "/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/share/lua/5.1/?.lua;/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/share/lua/5.1/?/init.lua;/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/lib/luarocks/rocks-5.1/?.lua;/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/lib/luarocks/rocks-5.1/?/init.lua"
local install_cpath_pattern = "/home/nix/.cache/nvim/packer_hererocks/2.1.0-beta3/lib/lua/5.1/?.so"
if not string.find(package.path, package_path_str, 1, true) then
  package.path = package.path .. ';' .. package_path_str
end

if not string.find(package.cpath, install_cpath_pattern, 1, true) then
  package.cpath = package.cpath .. ';' .. install_cpath_pattern
end

local function try_loadstring(s, component, name)
  local success, result = pcall(loadstring(s))
  if not success then
    print('Error running ' .. component .. ' for ' .. name)
    error(result)
  end
  return result
end

_G.packer_plugins = {
  ale = {
    config = { "require('config.ale')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/ale"
  },
  ["completion-nvim"] = {
    config = { "require('config.completions')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/completion-nvim"
  },
  delimitMate = {
    config = { "require('config.delimitmate')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/delimitMate"
  },
  fzf = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/fzf"
  },
  ["fzf.vim"] = {
    config = { "require('config.fzf')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/fzf.vim"
  },
  ["goyo.vim"] = {
    commands = { "Goyo" },
    config = { "require('config.goyo')" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/goyo.vim"
  },
  gruvbox = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/gruvbox"
  },
  ["markdown-preview.nvim"] = {
    commands = { "MarkdownPreview" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/markdown-preview.nvim"
  },
  ["nvim-bufferline.lua"] = {
    config = { "require('config.bufferline')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/nvim-bufferline.lua"
  },
  ["nvim-colorizer.lua"] = {
    config = { "require('config.colorizer')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/nvim-colorizer.lua"
  },
  ["nvim-treesitter"] = {
    config = { "require('config.treesitter')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/nvim-treesitter"
  },
  ["nvim-web-devicons"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/nvim-web-devicons"
  },
  ["onedark.vim"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/onedark.vim"
  },
  ["packer.nvim"] = {
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/packer.nvim"
  },
  ["rust.vim"] = {
    config = { "require('config.rust')" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/rust.vim"
  },
  ["startuptime.vim"] = {
    commands = { "StartupTime" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/startuptime.vim"
  },
  ["vim-afterglow"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-afterglow"
  },
  ["vim-bufkill"] = {
    commands = { "BD" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/vim-bufkill"
  },
  ["vim-commentary"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-commentary"
  },
  ["vim-floaterm"] = {
    config = { "require('config.floaterm')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-floaterm"
  },
  ["vim-lastplace"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-lastplace"
  },
  ["vim-localvimrc"] = {
    config = { "require('config.localvimrc')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-localvimrc"
  },
  ["vim-repeat"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-repeat"
  },
  ["vim-scriptease"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-scriptease"
  },
  ["vim-searchant"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-searchant"
  },
  ["vim-surround"] = {
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-surround"
  },
  ["vim-which-key"] = {
    config = { "require('config.which-key')" },
    loaded = true,
    path = "/home/nix/.local/share/nvim/site/pack/packer/start/vim-which-key"
  },
  vimtex = {
    config = { "require('config.vimtex')" },
    loaded = false,
    needs_bufread = false,
    path = "/home/nix/.local/share/nvim/site/pack/packer/opt/vimtex"
  }
}

-- Config for: delimitMate
require('config.delimitmate')
-- Config for: vim-which-key
require('config.which-key')
-- Config for: vim-floaterm
require('config.floaterm')
-- Config for: completion-nvim
require('config.completions')
-- Config for: vim-localvimrc
require('config.localvimrc')
-- Config for: ale
require('config.ale')
-- Config for: nvim-colorizer.lua
require('config.colorizer')
-- Config for: nvim-bufferline.lua
require('config.bufferline')
-- Config for: nvim-treesitter
require('config.treesitter')
-- Config for: fzf.vim
require('config.fzf')

-- Command lazy-loads
vim.cmd [[command! -nargs=* -range -bang -complete=file StartupTime lua require("packer.load")({'startuptime.vim'}, { cmd = "StartupTime", l1 = <line1>, l2 = <line2>, bang = <q-bang>, args = <q-args> }, _G.packer_plugins)]]
vim.cmd [[command! -nargs=* -range -bang -complete=file MarkdownPreview lua require("packer.load")({'markdown-preview.nvim'}, { cmd = "MarkdownPreview", l1 = <line1>, l2 = <line2>, bang = <q-bang>, args = <q-args> }, _G.packer_plugins)]]
vim.cmd [[command! -nargs=* -range -bang -complete=file BD lua require("packer.load")({'vim-bufkill'}, { cmd = "BD", l1 = <line1>, l2 = <line2>, bang = <q-bang>, args = <q-args> }, _G.packer_plugins)]]
vim.cmd [[command! -nargs=* -range -bang -complete=file Goyo lua require("packer.load")({'goyo.vim'}, { cmd = "Goyo", l1 = <line1>, l2 = <line2>, bang = <q-bang>, args = <q-args> }, _G.packer_plugins)]]

vim.cmd [[augroup packer_load_aucmds]]
vim.cmd [[au!]]
  -- Filetype lazy-loads
vim.cmd [[au FileType tex ++once lua require("packer.load")({'vimtex'}, { ft = "tex" }, _G.packer_plugins)]]
vim.cmd [[au FileType rust ++once lua require("packer.load")({'rust.vim'}, { ft = "rust" }, _G.packer_plugins)]]
vim.cmd("augroup END")
END

catch
  echohl ErrorMsg
  echom "Error in packer_compiled: " .. v:exception
  echom "Please check your config for correctness"
  echohl None
endtry

and this is my plugins.lua (also found in my dotfiles right here)

vim.cmd('packadd packer.nvim')

return require('packer').startup(function()
  -- Let Packer manage itself as an optional plugin
  use {'wbthomason/packer.nvim', opt = true}

  -- Lsp, completions and linting
  use {'neovim/nvim-lspconfig', config = [[require('config.lsp')]]}
  use {'nvim-lua/completion-nvim', config = [[require('config.completions')]]}
  use {'dense-analysis/ale', config = [[require('config.ale')]]}

  -- UI
  use {'norcalli/nvim-colorizer.lua', config = [[require('config.colorizer')]]}
  use {
    'nvim-treesitter/nvim-treesitter',
    config = [[require('config.treesitter')]],
  }
  use {
    'akinsho/nvim-bufferline.lua',
    requires = {
      'kyazdani42/nvim-web-devicons',
      config = [[require('config.devicons')]]
    },
    config = [[require('config.bufferline')]],
  }

  -- Scripting and profiling
  use 'tpope/vim-scriptease'
  use {'tweekmonster/startuptime.vim', cmd = 'StartupTime'}

  -- Ergonomics
  use {'Raimondi/delimitMate', config = [[require('config.delimitmate')]]}
  use 'tpope/vim-commentary'
  use 'tpope/vim-repeat'
  use 'tpope/vim-surround'

  -- File navigation
  use {
    'junegunn/fzf.vim',
    requires = {'junegunn/fzf'},
    config = [[require('config.fzf')]]
  }

  -- Filetype specific
  use {'rust-lang/rust.vim', ft = 'rust', config = [[require('config.rust')]]}
  use {'lervag/vimtex', ft = 'tex', config = [[require('config.vimtex')]]}

  -- Misc
  use {'liuchengxu/vim-which-key', config = [[require('config.which-key')]]}
  use {'voldikss/vim-floaterm', config = [[require('config.floaterm')]]}
  use {'embear/vim-localvimrc', config = [[require('config.localvimrc')]]}
  use 'farmergreg/vim-lastplace'
  use 'timakro/vim-searchant'
  use {'junegunn/goyo.vim', cmd = 'Goyo', config = [[require('config.goyo')]]}
  use {'qpkorr/vim-bufkill', cmd = 'BD'}
  use {
    'iamcco/markdown-preview.nvim',
    run = 'cd app && yarn install',
    cmd = 'MarkdownPreview'
  }

  -- Colorschemes
  use 'danilo-augusto/vim-afterglow'
  use 'morhetz/gruvbox'
  use 'joshdick/onedark.vim'
end)

it may seem that the plugin nvim-colorizer.lua is the problem, but if I comment that out I get the same exact error message, this time with nvim-treesitter. It seems that packer can't locate the modules it needs, possibly because of the unusual way NixOS file system is structured?

Finally, this is a video of the crash happening

https://user-images.githubusercontent.com/59321248/109642871-ca367280-7b53-11eb-93c9-4fc03cd6c163.mov

[wbthomason]

Thanks for the report! I'm not very familiar with NixOS, but the search paths in your error message don't look right. Another user recently mentioned that nix sometimes renames paths in a way that breaks things for this (https://github.com/wbthomason/packer.nvim/issues/235#issuecomment-786811424) - could you please send the output of tree ~/.local/share/nvim/site/pack, so that we can see if the paths there are as expected?

[noib3]

@wbthomason sure, here it is

$ tree ~/.local/share/nvim/site/pack
/home/nix/.local/share/nvim/site/pack
└── packer
    ├── opt
    │   └── packer.nvim
    │       ├── doc
    │       │   └── packer.txt
    │       ├── LICENSE
    │       ├── lua
    │       │   ├── packer
    │       │   │   ├── async.lua
    │       │   │   ├── clean.lua
    │       │   │   ├── compile.lua
    │       │   │   ├── display.lua
    │       │   │   ├── handlers.lua
    │       │   │   ├── install.lua
    │       │   │   ├── jobs.lua
    │       │   │   ├── load.lua
    │       │   │   ├── log.lua
    │       │   │   ├── luarocks.lua
    │       │   │   ├── plugin_types
    │       │   │   │   ├── git.lua
    │       │   │   │   └── local.lua
    │       │   │   ├── plugin_types.lua
    │       │   │   ├── plugin_utils.lua
    │       │   │   ├── result.lua
    │       │   │   ├── update.lua
    │       │   │   └── util.lua
    │       │   └── packer.lua
    │       └── README.md
    └── start

8 directories, 21 files

[wbthomason]

Wait, I'm sorry. I think I know what's happening here. Just to check, does :PackerInstall fail if you hit enter to get past the initial error messages when you open Neovim?

What's going on is that you already have a packer_compiled in place, but don't have any plugins installed. So, when you start Neovim, packer_compiled gets sourced and tries to load things that aren't there yet.

If PackerInstall is there after the errors, then just running that should work. Otherwise, we can figure out a way to make sure the install happens before packer_compiled is sourced.

[noib3]

Just to check, does :PackerInstall fail if you hit enter to get past the initial error messages when you open Neovim?

Yes, it fails in the exact same way if I just open neovim, hit enter to get past the errors and issue a :PackerInstall (basically what I did in the video minus the :PackerCompile step).

What's going on is that you already have a packer_compiled in place, but don't have any plugins installed. So, when you start Neovim, packer_compiled gets sourced and tries to load things that aren't there yet.

I'm not that's what's going on here. If I delete my packer_compiled I get no initial error when opening neovim, instead I get it after issuing a :PackerCompile. After that, :PackerInstall still crashes.

[wbthomason]

What happens if you:

1. Delete your packer_compiled
2. Open Neovim and just run PackerInstall
3. Then run PackerCompile

Running PackerCompile before things are installed will cause the errors you're seeing, because packer will source the compiled file. I admit I have no idea why Neovim is segfaulting when you run :PackerInstall after pushing past the errors, but I'm hopeful that avoiding the errors entirely will let the install work.

You can also run PackerSync to do the install and compile all together, in proper sequence.

[noib3]

It still segfaults when I run a PackerInstall, same with PackerSync. Deleting my packer_compiled gets rid of the errors, but I don't think it's got much to do with neovim segfaulting. That happens with PackerInstall, PackerUpdate and PackerSync.

[wbthomason]

Fascinating. I think the segfault is separate from the other errors, then.

Do you think you could get a core dump so we can figure out what's triggering the segfault? I wonder if this is an upstream issue.

[noib3]

Do you think you could get a core dump so we can figure out what's triggering the segfault?

Sure, how do I do that?

[wbthomason]

This has a pretty good overview: https://jvns.ca/blog/2018/04/28/debugging-a-segfault-on-linux/ (skipping the step about valgrind unless you feel like doing that too, but it's not necessary for a core dump)

[noib3]

@wbthomason here's the backtrace of the core dump:

(gdb) bt
#0  0x00007f5319f12751 in __strlen_avx2 () from /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libc.so.6
#1  0x00000000004fb657 in tv_dict_add_str ()
#2  0x00000000004dfb57 in f_environ.lto_priv ()
#3  0x0000000000503855 in call_func ()
#4  0x0000000000587144 in nlua_call ()
#5  0x00007f5319f849d5 in ?? () from /nix/store/jzyd44q72bg1agcsnr0khzyfj1nd9ijn-luajit-2.1.0-2020-12-28/lib/libluajit-5.1.so.2
#6  0x00007f5319f85825 in ?? () from /nix/store/jzyd44q72bg1agcsnr0khzyfj1nd9ijn-luajit-2.1.0-2020-12-28/lib/libluajit-5.1.so.2
#7  0x00007f5319fda159 in lua_pcall () from /nix/store/jzyd44q72bg1agcsnr0khzyfj1nd9ijn-luajit-2.1.0-2020-12-28/lib/libluajit-5.1.so.2
#8  0x00000000005879f1 in nlua_schedule_event ()
#9  0x0000000000505509 in multiqueue_process_events ()
#10 0x00000000005d4c4f in nv_event.lto_priv ()
#11 0x00000000005ca97e in normal_execute.lto_priv ()
#12 0x000000000068109d in state_enter ()
#13 0x00000000005c614b in normal_enter ()
#14 0x00000000004479e3 in main ()

[wbthomason]

Thanks, and sorry for the radio silence! This looks like it might be an issue with a non-null-terminated string? (given the crash seems to happen in strlen) Of course, I have no idea where such a malformed string might be coming from.

Do you have anything in ~/.cache/nvim/packer.nvim.log? That might help us narrow down where the install is failing.

[mjlbach]

This isn't due to packer but because colorizer.so is failing to load (probably due to cpath). A similar issue happens with sql.nvim which depends on sqlite3 https://github.com/tami5/sql.nvim/pull/36 https://github.com/tami5/sql.nvim/issues/28

[Congee]

Glad to see anothe nixos user ;)

Not sure if relevant, I am using home-manager by symlink-ing $XDG_CONFG_HOME/nvim via

xdg.configFile."nvim".source = ./config/nvim;

The problem is, usually, $XDG_CONFIG_HOME may not be writable if the runtimepath you are using is linked to a read-only overlay file system pointing to /nix/store. packer.nvim by default writes the packer_compiled.vim to $XDG_CONFIG_HOME via compile_path = util.join_paths(vim.fn.stdpath('config'), 'packer_compiled.vim').

To avoid this issue, simply use config.lib.file.mkOutOfStoreSymlink provided by home-manager when you do symlink-ing or set the compile_path to $XDG_DATA_HOME.

[wbthomason]

Thanks @mjlbach and @Congee for the additional context! I'm closing this for now, since it seems (1) not caused by packer and (2) solved by the fix @Congee shared, but please feel free to reopen if I've misunderstood.