Can't find lz4 and can't disable it

[mdorier]

I'm trying to build c-raft. I have lz4 installed and available for pkg-config to find:

$ pkg-config --libs --cflags liblz4
-I/projects/spack/opt/spack/linux-ubuntu22.04-sandybridge/gcc-11.3.0/lz4-1.9.4-64syhqtoz2e5finrv5ejiv3mlwu5bu5i/include -L/projects/spack/opt/spack/linux-ubuntu22.04-sandybridge/gcc-11.3.0/lz4-1.9.4-64syhqtoz2e5finrv5ejiv3mlwu5bu5i/lib -llz4

When I run c-raft's configure, it correctly finds it:

...
checking for LZ4... yes
...

However when I run make, I get the following:

  CC       src/libraft_la-byte.lo
  CC       src/libraft_la-client.lo
  CC       src/libraft_la-compress.lo
../src/compress.c:4:10: fatal error: lz4frame.h: No such file or directory
    4 | #include <lz4frame.h>
      |          ^~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:2120: src/libraft_la-compress.lo] Error 1

The lz4frame.h is in the include path of lz4.

Additionally, if I configure with --disable-lz4, the configure script will still find lz4, enable it, and fail just the same way.

The only way I can build c-raft currently is by (1) making sure that pkg-config cannot find lz4 (i.e. removing its path from PKG_CONFIG_PATH) and configure with --disable-lz4.

There is also an inconsistency in how lz4 and libuv are treated: if libuv is not found, configure will simply disable support for it (equivalent to adding --disable-uv). But if lz4 is not found, configure will fails, unless --disable-lz4 was explicitly set.

Note: tested with latest release (0.17.1) and master branch.

[cole-miller]

Thanks for the report. Could you try running make clean; make V=1 with the same setup, so I can inspect the failing cc invocation?

[mdorier]

Sure, here is the command that fails:

libtool: compile:  gcc -DPACKAGE_NAME=\"raft\" -DPACKAGE_TARNAME=\"raft\" -DPACKAGE_VERSION=\"0.17.1\" "-DPACKAGE_STRING=\"raft 0.17.1\"" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -DPACKAGE=\"raft\" -DVERSION=\"0.17.1\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DHAVE_WCHAR_H=1 -DSTDC_HEADERS=1 -D_ALL_SOURCE=1 -D_DARWIN_C_SOURCE=1 -D_GNU_SOURCE=1 -D_HPUX_ALT_XOPEN_SOCKET_API=1 -D_NETBSD_SOURCE=1 -D_OPENBSD_SOURCE=1 -D_POSIX_PTHREAD_SEMANTICS=1 -D__STDC_WANT_IEC_60559_ATTRIBS_EXT__=1 -D__STDC_WANT_IEC_60559_BFP_EXT__=1 -D__STDC_WANT_IEC_60559_DFP_EXT__=1 -D__STDC_WANT_IEC_60559_FUNCS_EXT__=1 -D__STDC_WANT_IEC_60559_TYPES_EXT__=1 -D__STDC_WANT_LIB_EXT2__=1 -D__STDC_WANT_MATH_SPEC_FUNCS__=1 -D_TANDEM_SOURCE=1 -D__EXTENSIONS__=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_STDIO_H=1 -DHAVE_ASSERT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_LINUX_IO_URING_H=1 -DHAVE_LINUX_AIO_ABI_H=1 -DHAVE_DECL_UV_FS_O_CREAT=0 -I. -I.. -std=c11 -g -fcf-protection --param=ssp-buffer-size=4 -pipe -fno-strict-aliasing -fdiagnostics-color -fexceptions -fstack-clash-protection -fstack-protector-strong -fasynchronous-unwind-tables -fdiagnostics-show-option -Wall -Wextra -Wpedantic -Wimplicit-fallthrough=5 -Wcast-align -Wstrict-prototypes -Wlogical-op -Wmissing-include-dirs -Wold-style-definition -Winit-self -Wfloat-equal -Wsuggest-attribute=noreturn -Wformat=2 -Wendif-labels -Wdate-time -Wnested-externs -Wconversion -Werror=implicit-function-declaration -Wunused-but-set-variable -Werror=return-type -Werror=incompatible-pointer-types -Wshadow -Werror=overflow -Werror=shift-count-overflow -Werror=shift-overflow=2 -Warray-bounds -Wrestrict -Wreturn-local-addr -Wstringop-overflow -fno-omit-frame-pointer -I/projects/spack/opt/spack/linux-ubuntu22.04-sandybridge/gcc-11.3.0/libuv-1.44.1-6lawhnbtqlrqejwerjpcaakrulmk6fvo/include -DNDEBUG -fvisibility=hidden -DLZ4_AVAILABLE -DLZ4_ENABLED -g -O2 -MT src/libraft_la-compress.lo -MD -MP -MF src/.deps/libraft_la-compress.Tpo -c ../src/compress.c  -fPIC -DPIC -o src/.libs/libraft_la-compress.o
../src/compress.c:4:10: fatal error: lz4frame.h: No such file or directory
    4 | #include <lz4frame.h>
      |          ^~~~~~~~~~~~
compilation terminated.

It looks like LZ4_AVAILABLE and LZ4_ENABLED are defined, but there is no cflags and ldflags for lz4.

If I look at config.log, I find this:

LZ4_AVAILABLE_FALSE='#'
LZ4_AVAILABLE_TRUE=''
LZ4_CFLAGS='-I/projects/spack/opt/spack/linux-ubuntu22.04-sandybridge/gcc-11.3.0/lz4-1.9.4-64syhqtoz2e5finrv5ejiv3mlwu5bu5i/include'
LZ4_ENABLED_FALSE='#'
LZ4_ENABLED_TRUE=''
LZ4_LIBS='-L/projects/spack/opt/spack/linux-ubuntu22.04-sandybridge/gcc-11.3.0/lz4-1.9.4-64syhqtoz2e5finrv5ejiv3mlwu5bu5i/lib -llz4'

So it looks like the configure script found lz4, but the necessary flags are not propagated to the compile commands.

[mdorier]

$(LZ4_CFLAGS) needs to be added in https://github.com/canonical/raft/blob/master/Makefile.am lines 87 and 88 to fix the problem.

This doesn't fix the problem of --disable-lz4 being ignored, though.

[cole-miller]

This is motivating me to finally rewrite our build system and do away with Autotools :). Thanks for digging into the issue, I'll have a PR up to fix this and #376 shortly.