Missing LibLUV dependency for most recent neovim versions

[rvolosatovs]

Issue description

Since https://github.com/neovim/neovim/pull/10123 neovim requires libluv for builds. See https://github.com/neovim/neovim/wiki/Following-HEAD#20190610

There's no released version yet containing this change, but it's nice to prepare for this already in order to simplify the update later.

The build on https://github.com/rvolosatovs/nixpkgs/commit/2b7a3bb484b9312ab500f106e50d3250f6866be7 fails for me with:

CMake Error at /nix/store/06zj4w9fdhwfpz9grbm01fqwgsl1ig4g-cmake-3.13.4/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /nix/store/06zj4w9fdhwfpz9grbm01fqwgsl1ig4g-cmake-3.13.4/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:34 (find_package_handle_standard_args)
  CMakeLists.txt:386 (find_package)

I have no experience with either CMake or Lua, so it's quite hard for me to reason about this and debug, but as far as I understand the issue is that build requires luv.h(https://github.com/luvit/luv/blob/1.30.0-0/src/luv.h ???) to be present at build time and luaPackages.luv does not contain it:

tree $(nix eval --raw nixpkgs.luaPackages.luv)
/nix/store/gkszplgfs3z05765pbqqq9ahhsv1j8wv-lua5.2-luv-1.29.1-2
├── lib
│   └── lua
│       └── 5.2
│           └── luv.so
├── luv-1.29.1-2-rocks
│   ├── luv
│   │   └── 1.29.1-2
│   │       ├── doc
│   │       │   ├── docs.md
│   │       │   ├── LICENSE.txt
│   │       │   └── README.md
│   │       ├── luv-1.29.1-2.rockspec
│   │       └── rock_manifest
│   └── manifest
└── nix-support
    └── propagated-build-inputs

Refs https://github.com/neovim/neovim/issues/10181 https://github.com/neovim/neovim/pull/10359#issuecomment-506741724

cc @manveru

[rvolosatovs]

Update of luv to version 1.30.0 did not help. https://github.com/NixOS/nixpkgs/compare/master...rvolosatovs:feature/nightly-neovim

[gloaming]

They didn't include a pkgconfig until after that version: https://github.com/luvit/luv/commit/df03f2ac6d4e7269cb0c88be3ef44cf1f0de9b24

You can specify it manually, but our version of luv does not include the header files, since that requires a different build configuration (see just after the pkgconfig part). Changing the build configuration for lua packages is apparently a problem? @dtzWill @Shados

https://github.com/NixOS/nixpkgs/blob/274715cbc355e9d50e07a2f60a65a183f7d9855d/pkgs/development/lua-modules/overrides.nix#L243-L256

[teto]

There is a lot going on between libluv's buildsystem and neovim's. It's urgent to wait as some say. https://github.com/neovim/neovim/pull/10449 is another way closer to a solution.

[blueyed]

@teto While a step to better integration, https://github.com/neovim/neovim/pull/10449 should only help when not using LuaJIT.

[gloaming]

@teto I don't think that's relevant to us in any case. That discussion is about how to build the "bundled" luv, which we aren't using anyway.

[Shados]

@gloaming I think we will want to wait until luvit/luv#344 and neovim/neovim#10292 both play out, and a new luv luarock has been published. If necessary, we can then just add a postInstall phase to luv's existing overrides, to perform the make BUILD_MODULE=0 install (with appropriate make flags).

[blueyed]

Waiting for https://github.com/luvit/luv/pull/344 is not really necessary, is it? Also, from what I can tell LuaRocks 3 has several issues (at least within Neovim's bundled build system, for both Linux/Windows)).

[Shados]

@blueyed nixpkgs internally uses Luarocks 3.1.3 currently, so if post-1.3.0 luv changes have caused issues building with Luarocks 3.1.3 in general (rather than just in neovim's bundled build system), then yes, we'd want to wait for you/someone else to figure that out :P.

More broadly though, I am interested in knowing what issues you've run into with Luarocks 3, at least on the Linux side.

EDIT: Feel free to poke me on Freenode (same nick) if you want to discuss.

[blueyed]

@Shados I see.. :)

For Linux it's only that we have to switch to "make install" instead of "make bootstrap". https://github.com/luarocks/luarocks/pull/1043 should fix this. That's a good work around I guess - I am not sure why "bootstrap" was used in the first place, or what benefits it would provide.

The (main) issue on Windows is fixed via https://github.com/luvit/luv/commit/9d28d38cfd0b6d966c061a35cf89b930364e228b (released already).

[teto]

Seems like there was a new release https://github.com/luvit/luv/commit/b9b096d143b38dc3eae5aea8db9aa7982c85bbb8 , might be worth trying again.

[rvolosatovs]

The issue persists on https://github.com/rvolosatovs/nixpkgs/commit/19e8bf796eeacfaa3d44818fa4cc2ccf029c0d20 (see diff here: https://github.com/NixOS/nixpkgs/compare/master...rvolosatovs:19e8bf796eeacfaa3d44818fa4cc2ccf029c0d20):

Build log

``` nix-build -A neovim these derivations will be built: /nix/store/h5h2qj4jg02br9qrkx6p43wfpvwfm7g8-ruby2.5.5-msgpack-1.2.6.drv /nix/store/q68c9gpk31906bp8xl2i40lrdam329wz-ruby2.5.5-neovim-0.8.0.drv /nix/store/7vd8yq0a5cl6hjh4rn18vd3hc98pl2q6-neovim-ruby-env.drv /nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv /nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv building '/nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv'... building '/nix/store/h5h2qj4jg02br9qrkx6p43wfpvwfm7g8-ruby2.5.5-msgpack-1.2.6.drv'... unpacking sources patching sources configuring no configure script, doing nothing installing buildFlags: unpacking sources unpacking source archive /nix/store/flslb5ngm36x22jdfjl5s93y7q931njq-source WARNING: You build with buildroot. Build root: / Bin dir: /nix/store/ws471an5m3ysi7mng7hlh99jj3p7270d-ruby2.5.5-msgpack-1.2.6/lib/ruby/gems/2.5.0/bin Gem home: /nix/store/ws471an5m3ysi7mng7hlh99jj3p7270d-ruby2.5.5-msgpack-1.2.6/lib/ruby/gems/2.5.0 Building native extensions. This could take a while... source root is source patching sources applying patch /nix/store/jnp9n2x9cdvl47da3656illrd3p2qvb5-system_rplugin_manifest.patch patching file runtime/autoload/remote/host.vim patching file runtime/plugin/rplugin.vim configuring cmake flags: -DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/include -DCMAKE_INSTALL_LIBDIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/lib -DCMAKE_INSTALL_NAME_DIR=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_DEPLOYMENT_TARGET= -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=last -DCMAKE_STRIP=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/strip -DCMAKE_RANLIB=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/ranlib -DCMAKE_AR=/nix/store/mgdjnsrkqgmxqjaqaxgqyqm7fwyi96fk-binutils-2.31.1/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly -DGPERF_PRG=/nix/store/xckpvzqf07fmsc666p89rybay2bi32d0-gperf-3.1/bin/gperf -DLUA_PRG=/nix/store/adjypv75qi2x8b5zhwg306sir901jiij-luajit-2.1.0-beta3-env/bin/lua -DUSE_BUNDLED_LUV=ON -- The C compiler identification is GNU 7.4.0 -- Check for working C compiler: /nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/gcc -- Check for working C compiler: /nix/store/hpzj855nkgjvg58nrhq4910sb9q3kss1-gcc-wrapper-7.4.0/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- CMAKE_INSTALL_PREFIX=/nix/store/m8c89zhvwl7c62zvlmgl1f9d8jsnzikc-neovim-unwrapped-nightly -- CMAKE_BUILD_TYPE=Release -- MIN_LOG_LEVEL not specified, default is 1 (INFO) -- Replacing -O3 in CMAKE_C_FLAGS_RELEASE with -O2 -- Performing Test HAS_OG_FLAG -- Performing Test HAS_OG_FLAG - Success -- Performing Test HAS_ACCEPTABLE_FORTIFY -- Performing Test HAS_ACCEPTABLE_FORTIFY - Success -- Performing Test HAVE_EXECINFO_BACKTRACE -- Performing Test HAVE_EXECINFO_BACKTRACE - Success -- Performing Test HAVE_BUILTIN_ADD_OVERFLOW -- Performing Test HAVE_BUILTIN_ADD_OVERFLOW - Success -- Performing Test HAS_WIMPLICIT_FALLTHROUGH_FLAG -- Performing Test HAS_WIMPLICIT_FALLTHROUGH_FLAG - Success -- Performing Test HAS_WVLA_FLAG -- Performing Test HAS_WVLA_FLAG - Success -- Performing Test HAS_FSTACK_PROTECTOR_STRONG_FLAG -- Performing Test HAS_FSTACK_PROTECTOR_STRONG_FLAG - Success -- Performing Test HAS_FSTACK_PROTECTOR_FLAG -- Performing Test HAS_FSTACK_PROTECTOR_FLAG - Success -- Performing Test HAS_DIAG_COLOR_FLAG -- Performing Test HAS_DIAG_COLOR_FLAG - Success -- Found PkgConfig: /nix/store/01a3dppfi3rv03m8yx2nwxbkklggyzf9-pkg-config-0.29.2/bin/pkg-config (found version "0.29.2") -- Looking for dlopen in dl -- Looking for dlopen in dl - found -- Looking for kstat_lookup in kstat -- Looking for kstat_lookup in kstat - not found -- Looking for kvm_open in kvm -- Looking for kvm_open in kvm - not found -- Looking for gethostbyname in nsl -- Looking for gethostbyname in nsl - found -- Looking for perfstat_cpu in perfstat -- Looking for perfstat_cpu in perfstat - not found -- Looking for clock_gettime in rt -- Looking for clock_gettime in rt - found -- Looking for sendfile in sendfile -- Looking for sendfile in sendfile - not found -- Found LibUV: /nix/store/43gl6cfmzagx5plly02hswb7yhrl9fyn-libuv-1.30.1/lib/libuv.so (Required is at least version "1.28.0") -- Found Msgpack: /nix/store/823rqv68q4aw29k1kl71kx5b7vfcs7xb-msgpack-3.0.1/lib/libmsgpackc.so (found suitable version "3.0.1", minimum required is "1.0.0") CMake Error at /nix/store/y09hh3g31pnm68cfb6rnn5pql04si6mp-cmake-3.14.5/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:137 (message): Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR) (Required is at least version "1.30.0") Call Stack (most recent call first): /nix/store/y09hh3g31pnm68cfb6rnn5pql04si6mp-cmake-3.14.5/share/cmake-3.14/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) cmake/FindLibLUV.cmake:34 (find_package_handle_standard_args) CMakeLists.txt:375 (find_package) -- Configuring incomplete, errors occurred! See also "/build/source/build/CMakeFiles/CMakeOutput.log". See also "/build/source/build/CMakeFiles/CMakeError.log". builder for '/nix/store/ngrp0rnwpi1js3qyyw6pzf1qf1pax5l2-neovim-unwrapped-nightly.drv' failed with exit code 1 cannot build derivation '/nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv': 1 dependencies couldn't be built error: build of '/nix/store/4zibm78fbh75h4rqvqxy9y2ykkrs5psz-neovim-nightly.drv' failed ```

[gloaming]

@rvolosatovs could you kindly wrap long logs in

? :)

Well, nothing relevant changed upstream, so it's not surprising it doesn't work.

[rvolosatovs]

@gloaming updated

[gloaming]

I have it working here: https://github.com/gloaming/nixpkgs/commit/930254677c19e628ec2ab0033758cf67e0944925

To do it more nicely, we would need luv to provide a unified build configuration that produces all outputs and a single pkgconfig (and rockspec, I assume).

Preferably, we should also get the Neovim build to accept a dynamic library.

[blueyed]

Preferably, we should also get the Neovim build to accept a dynamic library.

Are you referring to Neovim itself, or your build of it? (Neovim itself should not require a static library - it only does so internally when using its bundled deps / third-party build, but that should not be used usually with distributions in the first place)

I have this for example (in CMakeCache.txt): LIBLUV_LIBRARY:FILEPATH=/usr/lib/libluv.so.

[gloaming]

Ah. The problem is that our library is named luv.so instead of libluv.so, but Neovim's CMake does not pass the actual filename to gcc:

gcc [...] -L/nix/store/ywhwnh7x6yi331d7x46s8k4wnxgqnkdm-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1 -lluv -luv -lrt -lpthread -lnsl -ldl -ldl -lnsl -lrt -lmsgpackc -lvterm -ltermkey -lunibilium -lpthread -lm -lutil -lluajit-5.1 -lpthread -lmsgpackc -lvterm -ltermkey -lunibilium -lpthread -lm -lutil -lluajit-5.1
/nix/store/glhymgljg23hkyii6dgc5b276ajafm2x-binutils-2.31.1/bin/ld: cannot find -lluv
collect2: error: ld returned 1 exit status
make[2]: *** [src/nvim/CMakeFiles/nvim.dir/build.make:3218: bin/nvim] Error 1

So it implicitly requires that the file is called libluv.so. I think that's a Neovim bug.

[blueyed]

The problem is that our library is named luv.so instead of libluv.so

Why?

Neovim's CMake looks for "luv" as the library name, so this translates to "libluv.so" then.

You could use -DLIBLUV_LIBRARY:FILEPATH=/path/to/luv.so I guess, but I wonder why not using libluv.so from the beginning.

[teto]

@gloaming your patch works thanks !

I think nixpkgs' luv.so should be named libluv.so as it's the convention. Why isn't it the case already? Is that an issue with luv's installer ?

NB: I merged the libluv 1.30 bump thanks

[gloaming]

I checked several other Lua libraries and they are all named the same way. It appears to be the convention:

https://github.com/brimworks/lua-zip/blob/e01fac77e7ebf1feceed8f95a5de96aa2758e06f/CMakeLists.txt#L34-L37 https://github.com/libmpack/libmpack-lua/commit/1009ed35583333cfe8733cd99c34b8ccfe94df6d

[vcunat]

Yes, that's so. Also, I've never seen trying to link a lua library the normal C way – they're always open dynamically by require('luv') (or an equivalent to this lua code).

[blueyed]

Sure require('luv') makes sense. Also the include location is "luv" for me on Arch Linux:

libluv /usr/
libluv /usr/include/
libluv /usr/include/luv/
libluv /usr/include/luv/lhandle.h
libluv /usr/include/luv/lreq.h
libluv /usr/include/luv/luv.h
libluv /usr/include/luv/util.h
libluv /usr/lib/
libluv /usr/lib/libluv.so
libluv /usr/lib/libluv.so.1
libluv /usr/lib/libluv.so.1.30.0

According to https://cmake.org/cmake/help/latest/command/find_library.html CMake should find it though:

Each library name given to the NAMES option is first considered as a library file name and then considered with platform-specific prefixes (e.g. lib) and suffixes (e.g. .so). Therefore one may specify library file names such as libfoo.a directly. This can be used to locate static libraries on UNIX-like systems.

Code ref in Neovim: https://github.com/neovim/neovim/blob/6e01ed6a4c859ff915b42422622f1ba3cc80efd5/cmake/FindLibLUV.cmake#L23-L26

[gloaming]

It does find it: -- Found LibLUV: /nix/store/3vq3irgvpx00cvp7y8aay64vk7rkwbdl-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/luv.so (Required is at least version "1.30.0")

The problem is the linker options passed to gcc. It should be -l:luv.so instead of -lluv, see ld(1):

-l namespec --library=namespec Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.

On systems which support shared libraries, ld may also search for files other than libnamespec.a. Specifically, on ELF and SunOS systems, ld will search a directory for a library called libnamespec.so before searching for one called libnamespec.a. (By convention, a ".so" extension indicates a shared library.) Note that this behavior does not apply to :filename, which always specifies a file called filename.

[blueyed]

Ah, I see. I would assume CMake to handle that.. we're using target_link_libraries here: https://github.com/blueyed/neovim/blob/e107f94ea2e8dd4e664a7598a69eb0dd1b1c5f09/src/nvim/CMakeLists.txt#L411. According to https://cmake.org/cmake/help/latest/command/target_link_libraries.html:

There are some cases where CMake may ask the linker to search for the library (e.g. /usr/lib/libfoo.so becomes -lfoo), such as when a shared library is detected to have no SONAME field. See policy CMP0060 for discussion of another case.

I assume we could set the policy to new maybe? https://cmake.org/cmake/help/latest/policy/CMP0060.html#policy:CMP0060

Having a SONAME might be a better workaround though?

[blueyed]

Can you try the branch from https://github.com/neovim/neovim/pull/10661?

[gloaming]

No change; it looks like CMake is determined that it knows better in the case where a library has no SONAME. This is annoying for us, since under Nix, packages are not relocatable by definition, so there's no reason to try and prevent it. I think ideally we would get CMake to provide a way to disable this behaviour.

On the other hand, I don't know whether there's any particular reason that our Lua libraries don't have a SONAME in the first place. Presumably this is standard Lua behaviour and nobody needed it since they weren't linking in the first place.

[blueyed]

According to https://cmake.org/cmake/help/latest/policy/CMP0060.html#policy:CMP0060 it should use the new behavior always then. I assume you have CMake >= 3.3? Another option might be to use IMPORTED as mentioned with the policy. Can you add a message(STATUS ${NVIM_EXEC_LINK_LIBRARIES}) to check that it is still absolute there, and that the policy gets set actually (i.e. put it in the if block there).

[blueyed]

I think I found a fix - can you re-test the branch (force-pushed), please?

[blueyed]

Sorry, that is about finding again only again.

However, when using CMP0060 additionally, it then links fine. But fails when starting:

build/bin/nvim: error while loading shared libraries: libluv.so.1: cannot open shared object file: No such file or directory

Without the policy enabled I see what you get:

/usr/bin/ld: cannot find -lluv

Strange..

[blueyed]

That seems to be due to the name hardcoded in the ".so"? Having it as "libluv.so.1", with a symlink "luv.so" to it works then.

[gloaming]

Yes, we're on CMake 3.14.5.

I don't think it says CMP0060 will cause it to always happen, see here: https://cmake.org/cmake/help/latest/command/target_link_libraries.html#command:target_link_libraries

There are some cases where CMake may ask the linker to search for the library (e.g. /usr/lib/libfoo.so becomes -lfoo), such as when a shared library is detected to have no SONAME field. See policy CMP0060 for discussion of another case.

I forced the build by manually passing linker flags, and I noticed that the resulting binary in our case has the expected reference:

 0x0000000000000001 (NEEDED)             Shared library: [luv.so]

(Although it doesn't acquire the RUNPATH for some reason.)

---

In any case, it looks like this is not a NixOS-specific issue - it will arise on any distribution that doesn't rename the lua shared object. I think you guys need to take this up with CMake and the Lua build system to figure out the right way forward.

[blueyed]

Have you tried the latest branch? (from yesterday)

luv explicitly requests to have no prefix here with -DBUILD_MODULE=1: https://github.com/luvit/luv/blob/d7e0afb3f35877fde31e0f31e001388013310fb5/CMakeLists.txt#L133

This is what the Arch User Repo package is using:

build() {
    mkdir "${srcdir}/build"
    cd "${srcdir}/build"
    cmake -DWITH_SHARED_LIBUV=ON -DLUA_BUILD_TYPE=System \
        -DBUILD_MODULE=OFF -DBUILD_SHARED_LIBS=ON \
        -DCMAKE_INSTALL_PREFIX=/usr \
        "${srcdir}/luv-${pkgver}-${pkgrel}"
    make
}

package() {
    cd "${srcdir}/build"
    DESTDIR="${pkgdir}" make install
}

Maybe you could combine it / build it twice to have both variants?

As far as I can see luv.so is only useful for Lua, and there it would go into a non-standard location (/usr/lib/lua/5.3/luv.so) then.

I forced the build by manually passing linker flags

How exactly?

For reference, this is the CMake code:

bool cmComputeLinkInformation::CheckImplicitDirItem(std::string const& item)
{
  // We only switch to a pathless item if the link type may be
  // enforced.  Fortunately only platforms that support link types
  // seem to have magic per-architecture implicit link directories.
  if (!this->LinkTypeEnabled) {
    return false;
  }

  // Check if this item is in an implicit link directory.
  std::string dir = cmSystemTools::GetFilenamePath(item);
  if (this->ImplicitLinkDirs.find(dir) == this->ImplicitLinkDirs.end()) {
    // Only libraries in implicit link directories are converted to
    // pathless items.
    return false;
  }

  // Only apply the policy below if the library file is one that can
  // be found by the linker.
  std::string file = cmSystemTools::GetFilenameName(item);
  if (!this->ExtractAnyLibraryName.find(file)) {
    return false;
  }

  // Check the policy for whether we should use the approach below.
  switch (this->Target->GetPolicyStatusCMP0060()) {
    case cmPolicies::WARN:
      if (this->CMP0060Warn) {
        // Print the warning at most once for this item.
        std::string const& wid = "CMP0060-WARNING-GIVEN-" + item;
        if (!this->CMakeInstance->GetPropertyAsBool(wid)) {
          this->CMakeInstance->SetProperty(wid, "1");
          this->CMP0060WarnItems.insert(item);
        }
      }
    case cmPolicies::OLD:
      break;
    case cmPolicies::REQUIRED_ALWAYS:
    case cmPolicies::REQUIRED_IF_USED:
    case cmPolicies::NEW:
      return false;
  }

  // Many system linkers support multiple architectures by
  // automatically selecting the implicit linker search path for the
  // current architecture.  If the library appears in an implicit link
  // directory then just report the file name without the directory
  // portion.  This will allow the system linker to locate the proper
  // library for the architecture at link time.
  this->AddUserItem(file, false);

  // Make sure the link directory ordering will find the library.
  this->OrderLinkerSearchPath->AddLinkLibrary(item);

  return true;
}

In any case, it looks like this is not a NixOS-specific issue - it will arise on any distribution that doesn't rename the lua shared object.

Well, it depends on the usage of BUILD_MODULE=1.

This is from luv's README:

#### Build as shared library

If you want to build luv as a shared library run make with:

~/Code/luv> BUILD_MODULE=OFF BUILD_SHARED_LIBS=ON make

[gloaming]

Oh, now I get it. Other distros are providing a libluv installable outside of the Lua ecosystem, right?

I tried blueyed/neovim@8172f6ffb, it still gives the same error (cannot link -lluv). I am fairly sure the CMake documentation is not lying on this one.

How exactly?

https://github.com/gloaming/neovim/commit/4f06dd49f3b6a3982ebb62ef99f189350452fdd4 Although it works, and isn't even that much of a hack, I don't think it's reasonable to expect packages to jump through hoops like this. I think it's best to just provide a libluv output packages can consume and be done with it: https://github.com/gloaming/nixpkgs/compare/081abe540df6d30d1595dbf88cadcbcc752bcc0e...8e0f1358c27315f2a6d765da59cd0f64bf5d3c12

[gloaming]

On what platform is there still an actual distinction between SHARED and MODULE libraries? I don't think there is one. It seems bizarre to provide multiple copies of the library just to get a different filename and SONAME. But improving this situation is a question of whether anyone wants to overhaul luv's build script.

[vcunat]

IIRC on darwin they differ in the extension (.so and .dylib).

[blueyed]

This works:

make CMAKE_EXTRA_FLAGS='-DLIBLUV_LIBRARY:STRING=-Wl,/path/to/luv.so'

I.e. set our/nvim's LIBLUV_LIBRARY to a string value, which passes on an argument for the linker directly. This skips version checks etc then though, but that's ok when you know what you are doing.

[blueyed]

I think I've found a workaround: https://github.com/neovim/neovim/pull/10661#issuecomment-519392763 - please try the branch there again.

[blueyed]

Gentle ping to provide feedback on https://github.com/neovim/neovim/pull/10661.

[gloaming]

Yeah, it works :+1:

Note that Nix needs the second build configuration to get the headers anyway, so I think it's best for us to go that route. (Which means the changes on your end would be for other users' benefit, not us.)

Unfortunately the libluv build fails with the new luv release:

CMake Error: File /build/luv-1.30.1-0/luv-1.30.1-0/libluv.pc.in does not exist.
CMake Error at CMakeLists.txt:206 (configure_file):
  configure_file Problem configuring file

-- Configuring incomplete, errors occurred!
See also "/build/luv-1.30.1-0/luv-1.30.1-0/build.luarocks/CMakeFiles/CMakeOutput.log".

Error: Build error: Failed cmake.

builder for '/nix/store/5d36r8yvv4aklr5fysb3ph00hv4mjjjl-luajit-2.1.0-beta3-luv-1.30.1-0.drv' failed with exit code 1
error: build of '/nix/store/5d36r8yvv4aklr5fysb3ph00hv4mjjjl-luajit-2.1.0-beta3-luv-1.30.1-0.drv' failed

Someone should look into that.

[rvolosatovs]

FTR, latest nightly(https://github.com/neovim/neovim/commit/d7aea13fee879a5e7854f2ebe9b7321cd8daf84f) works fine for me on https://github.com/rvolosatovs/nixpkgs/tree/feature/nightly-neovim

[tex]

This works for me too.

[garbas]

works for me as well (commit).

[teto]

2 or 3 days ago, libvterm deps got updated so neovim 0.4 will need that libvterm https://github.com/teto/home/blob/4f4a3bc8a2bdd69bcbd2e3f78bcd72e8addd771d/config/nixpkgs/overlays/neovim.nix#L155 as well

[ecklf]

New release version (0.4.2) will now fail building because of missing Edit: (this is brew not nix btw sorry)

LIBLUV_LIBRARY / LIBLUV_INCLUDE_DIR

==> cmake .. -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG -DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/neovim/0.4.2 -DCMA
Last 15 lines from /Users/lynx/Library/Logs/Homebrew/neovim/04.cmake:
-- Looking for sendfile in sendfile - not found
-- Found LibUV: /usr/local/lib/libuv.dylib (Required is at least version "1.28.0")
-- Found Msgpack: /usr/local/lib/libmsgpackc.dylib (found suitable version "3.2.0", minimum required is "1.0.0")
CMake Error at /usr/local/Cellar/cmake/3.15.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.15.3/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:29 (find_package_handle_standard_args)
  CMakeLists.txt:375 (find_package)

[rvolosatovs]

@impulse this is what is necessary to build 0.4.2: https://github.com/NixOS/nixpkgs/pull/68882

[doronbehar]

Hey guys, I'm trying to build a specific branch of Neovim in a git checkout in order to check a certain Neovim PR. I used neovim-unwrapped's default.nix as a reference for my shell.nix (I use direnv) and I get the following error:

/var/src/neovim/src/nvim/lua/executor.c:35:10: fatal error: luv/luv.h: No such file or directory
 #include "luv/luv.h"
          ^~~~~~~~~~~
compilation terminated.
make[2]: *** [src/nvim/CMakeFiles/nvim.dir/build.make:567: src/nvim/auto/lua/executor.c.generated.h] Error 1
make[1]: *** [CMakeFiles/Makefile2:3433: src/nvim/CMakeFiles/nvim.dir/all] Error 2

This error appears although Cmake runs fine. I use cmakeFlags just as we use in neovim-unwrapped's default.nix.

I tried to test that branch's build by directly editing neovim-unwrapped's default.nix in my checkout of nixpkgs and the error didn't appear. If you are curios, the build wasn't totally successful but I've had other errors with it which seem to "advance" this one. This is my shell.nix:

with import <nixpkgs> {};
pkgs.mkShell rec {
  neovimLuaEnv = lua.withPackages(ps:
    (with ps; [
      lpeg
      luabitop
      mpack
      nvim-client
      luv
      coxpcall
      busted
      luafilesystem
      penlight
      inspect
      luaffi
    ])
  );
  cmakeFlags = [
    "-DGPERF_PRG=${gperf}/bin/gperf"
    "-DLUA_PRG=${neovimLuaEnv.interpreter}"
    "-DPREFER_LUA=ON"
    "-DBUSTED_PRG=${neovimLuaEnv}/bin/busted"

    "-DLIBLUV_LIBRARY=${lua.pkgs.luv}/lib/lua/${lua.luaversion}/luv.so"

    # This isn't part of neovim-unwrapped's default.nix but it's my attempt to
    # fix the missing header file as that's where I've found that luv.h
    "-DLIBLUV_INCLUDE_DIR=${neovimLuaEnv}/include"
  ];
  buildInputs = [
    gettext
    gperf
    libtermkey
    libuv
    libvterm
    luaPackages.luv
    msgpack
    ncurses
    neovimLuaEnv
    unibilium
    jemalloc
    libiconv
    procps
    glibcLocales
    cmake
    pkgconfig
  ];
}

BTW, I expected this issue to be closed now, since the 0.4.2 upgrade was merged. Are there any other luv related issues besides mine people are having?

[gloaming]

@doronbehar You've got luv in your buildInputs, should be lua.pkgs.luv.libluv

[doronbehar]

Thanks @gloaming, you've eventually helped me to find the problem - my channels were'nt updated enough and that's why lua.pkgs.luv.libluv gave me a missing attribute error.

I've also workaround any other issues with a much simpler shell.nix:

with import <nixpkgs> {};
pkgs.mkShell rec {
  cmakeFlags = neovim-unwrapped.cmakeFlags;
  buildInputs = neovim-unwrapped.buildInputs ++ neovim-unwrapped.nativeBuildInputs;
}

Anyway, to conclude, I don't experience any other issues related to luv so as I said before, I think this issue can be closed.

[vegerot]

Similar error in macOS Catalina when trying to run cmake -G Xcode neovim

-- Found LibUV: /usr/local/Cellar/libuv/1.31.0/lib/libuv.dylib (Required is at least version "1.28.0")
-- Found Msgpack: /usr/local/Cellar/msgpack/3.2.0/lib/libmsgpackc.dylib (found suitable version "3.2.0", minimum required is "1.0.0")
CMake Error at /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find LibLUV (missing: LIBLUV_LIBRARY LIBLUV_INCLUDE_DIR)
  (Required is at least version "1.30.0")
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.15.5/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/FindLibLUV.cmake:29 (find_package_handle_standard_args)
  CMakeLists.txt:371 (find_package)

And when trying to run "make ."

config.status: linking /Users/maxcoplan/Documents/workspace/neovim/.deps/build/src/libuv/test/fixtures/empty_file to test/fixtures/empty_file
config.status: linking /Users/maxcoplan/Documents/workspace/neovim/.deps/build/src/libuv/test/fixtures/load_error.node to test/fixtures/load_error.node
config.status: executing depfiles commands
config.status: executing libtool commands
ninja: build stopped: subcommand failed.
gmake: *** [Makefile:101: deps] Error 1

[doronbehar]

@vegerot is this the error you get when running cmake from nix-shell / direnv? What is your shell.nix?

[ersinakinci]

I'm getting the following error when trying to run nix-env -f '<nixpkgs>' -iA neovim on macOS--is it related? If so, how can I work around or fix it?

installing 'neovim-0.4.3'
these derivations will be built:
  /nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv
  /nix/store/5cswwdn7ljsnxy1xayvqvxyvqgxjcjkv-neovim-ruby-env.drv
  /nix/store/cz124m6x406nxhvyc3hvs3ffn9prck32-python-2.7.17-env.drv
  /nix/store/gw6080mnwgys53h0sw43bp26xksbx33k-python3-3.7.6-env.drv
  /nix/store/nr8zhi7hjf0g0rgvnknkn02grdfk3lyx-neovim-unwrapped-0.4.3.drv
  /nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv
building '/nix/store/5cswwdn7ljsnxy1xayvqvxyvqgxjcjkv-neovim-ruby-env.drv'...
created 12 symlinks in user environment
building '/nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv'...
unpacking sources
unpacking source archive /nix/store/iqsk16lkjsixwmrq9w3b6szh4nm3mpcf-luv-1.30.0-0.src.rock

Done. You may now enter directory
luv-1.30.0-0/luv-1.30.0-0
and type 'luarocks make' to build.
source root is ./luv-1.30.0-0/luv-1.30.0-0
setting SOURCE_DATE_EPOCH to timestamp 1561806879 of file ./luv-1.30.0-0/luv-1.30.0-0/src/work.c
patching sources
configuring
building
patching script interpreter paths in .
./deps/lua-compat-5.3/tests/test.lua: interpreter directive changed from "/usr/bin/env lua" to "/nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/bin/lua"
installing

luv 1.30.0-0 depends on lua >= 5.1 (5.1-1 provided by VM)
Warning: unmatched variable LUA_LIBDIR
-- The C compiler identification is Clang 7.1.0
-- The ASM compiler identification is Clang
-- Found assembler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang
-- Check for working C compiler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang
-- Check for working C compiler: /nix/store/2xv7ha3rsc8cd06hyajmhys91fnir3xg-clang-wrapper-7.1.0/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found LIBUV: /nix/store/7skf0l31yzjxsnw9wymjd93sl3yr69dg-libuv-1.34.2/lib/libuv.dylib
-- Lua: using information from luarocks
-- LUA_LIBDIR:
-- LUA_INCDIR: /nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/include/luajit-2.1
-- LUA: /nix/store/h3wc33sf3snh2whhn1q5n428kw2j6l1z-luajit-2.1.0-beta3/bin/luajit
-- Lua library: LUA_LIBRARIES-NOTFOUND
-- Configuring done
CMake Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   luv

This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /tmp/nix-build-luajit-2.1.0-beta3-luv-1.30.0-0.drv-0/luv-1.30.0-0/luv-1.30.0-0/build.luarocks
Scanning dependencies of target luv
[ 50%] Building C object CMakeFiles/luv.dir/src/luv.c.o
[100%] Linking C shared library libluv.dylib
[100%] Built target luv
[100%] Built target luv
Install the project...
-- Install configuration: ""
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.30.0.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.dylib
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/luv.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/util.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lhandle.h
-- Installing: /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/include/luv/lreq.h
cp: cannot stat '/nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib': No such file or directory

Error: Failed copying /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/luv-1.30.0-0-rocks/luv/1.30.0-0/lib/libluv.1.dylib to /nix/store/bf7vv7axnj3ispv4ac6xaf427zncrj3n-luajit-2.1.0-beta3-luv-1.30.0-0/lib/lua/5.1/libluv.1.dylib
builder for '/nix/store/479s9v0v0ma0wl3157hq6ppnvxk5ykzb-luajit-2.1.0-beta3-luv-1.30.0-0.drv' failed with exit code 1
cannot build derivation '/nix/store/nr8zhi7hjf0g0rgvnknkn02grdfk3lyx-neovim-unwrapped-0.4.3.drv': 1 dependencies couldn't be built
building '/nix/store/cz124m6x406nxhvyc3hvs3ffn9prck32-python-2.7.17-env.drv'...
cannot build derivation '/nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/851qma1nnxd1wbviwr3rayqmpkl5bi43-neovim-0.4.3.drv' failed