CMake + static library + static dependencies

[alexey-lysiuk]

I was experimenting with coelckers/gzdoom#1401, and encountered the problem with ZMusic static library that uses explicit linking to dependencies, i.e. with dynamic loading turned off.

Here is the set of commands I did on Ubuntu 20.04 using current HEAD at 95e6bfcd8cd219eee85588cf1e24781f5fd45e38.

git clone https://github.com/coelckers/ZMusic.git
cd ZMusic
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$(pwd)/../build_install -DBUILD_SHARED_LIBS=OFF -DDYN_SNDFILE=OFF -DDYN_MPG123=OFF -DDYN_FLUIDSYNTH=OFF ..
make install
cd ../..
git clone -b cmake-overhaul https://github.com/Blzut3/gzdoom.git
cd gzdoom
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH=$(pwd)/../../ZMusic/build_install ..

Configuration of GZDoom failed with the following output.

-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using system bzip2 library, includes found at /usr/include
-- Using system zlib, includes found at /usr/include
-- Using system jpeg library, includes found at /usr/include
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
CMake Error at /usr/share/cmake-3.16/Modules/CMakeFindDependencyMacro.cmake:47 (find_package):
  By not providing "FindSndFile.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "SndFile", but
  CMake did not find one.

  Could not find a package configuration file provided by "SndFile" with any
  of the following names:

    SndFileConfig.cmake
    sndfile-config.cmake

  Add the installation prefix of "SndFile" to CMAKE_PREFIX_PATH or set
  "SndFile_DIR" to a directory containing one of the above files.  If
  "SndFile" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  /home/user/ZMusic/build_install/lib/cmake/ZMusic/ZMusicConfig.cmake:35 (find_dependency)
  libraries/CMakeLists.txt:63 (find_package)

-- Configuring incomplete, errors occurred!
See also "/home/user/gzdoom/build/CMakeFiles/CMakeOutput.log".
See also "/home/user/gzdoom/build/CMakeFiles/CMakeError.log".

How should a modern way of locating dependencies (by importing CMake targets) work in this case? The current release of libsndfile can be built via CMake in order to produce a module that can be imported. This is not the case for FluidSynth that does not create a module during CMake build.

The mentioned issue became even more severe when I wanted to create a single self-contained GZDoom binary with all dependencies built as static libraries. For example, importing module of libsndfile tries to pull all its prerequisites (ogg, vorbis, flac, opus) via CMake ignoring other ways of discovery like pkg-config. At least, the given libraries supports creation of modules in their CMake build systems. However, dependencies of FluidSynth are not.

I'm not quite familiar with modern CMake, so I put it here in order to get an advise or maybe a solution of some sort.

[Blzut3]

I see you're doing an separate ZMusic compile instead of dropping the source tree into libraries/zmusic and letting it build monolithically. I first want to ask if this is a regression or something you just happened to run into while trying to do this?

In this case the problem that the logs are pointing to is that you'll need to add ZMusic's cmake/ directory to your CMAKE_MODULE_PATH so that you can use the find modules ZMusic provides. This will probably be enough to solve your issue judging by the commands you used to build ZMusic. If you had to manually provide the locations of SndFile et al, then you would probably need to copy those settings over to the GZDoom build since I don't think they would be baked into the ZMusic module, but this doesn't appear to be the case for you.

As an aside, on a more general note I want to state what "modern CMake" is, and it's simply the use of properties on targets to specify usage requirements instead of leaving it up to the consumer to do things like specify the include directories. This stronger abstraction leads to less spaghetti code, reduces the need for global state, and provides a way that you can pop the hood if the options provided aren't enough. Specifically you can write those Config.cmake/Find.cmake files manually for your specific setup. These don't need to be anything fancy but basically just add_library, set properties, and then an another add_library call to set the alias. Basically something like these lines: https://github.com/coelckers/ZMusic/blob/master/cmake/FindSndFile.cmake#L30-L37 Of course the benefit becomes more clear if you want to for example do different things if it's a debug vs release build in Visual Studio which isn't well supported with just the two standard variables.

[alexey-lysiuk]

Yes, it's a regression. There is no need to specify any paths (include, lib, CMake module) with the current "old" CMake build system. The only thing that requires special handling is the 'everything static' case. However, it's only about addition of a few libraries (that were not pulled automatically being dependency-of-dependency) to linker command line.

The suggestion about CMAKE_MODULE_PATH seems to work, configuration step succeeded at least. I didn't have a chance to check it in details. In general, I found this a bit annoying that to use new/modern/proper way of using CMake I should do additional steps than with old/obsolete way.

[Blzut3]

But is it really more steps? If I do the steps indicated in the original post with the current master of both projects I get this.

[100%] Linking CXX executable ../gzdoom
/usr/bin/ld: /home/blzut3/Code/ZMusic/build_install/lib/libzmusic.a(music_alsa_state.cpp.o): undefined reference to symbol 'snd_seq_query_next_port@@ALSA_0.9'
/usr/bin/ld: /lib/x86_64-linux-gnu/libasound.so.2: error adding symbols: DSO missing from command line

Which is caused by the fact that static libraries don't carry dependency information (fix that and it will complain about sndfile and so forth), so GZDoom doesn't know that it needs to link to all the things that ZMusic links to. Of course you can work around this by changing ZMUSIC_LIBRARIES to manually include all the libraries that it linked to.

On the other hand this works fine for me with the ZMusic built exactly like you did:

cmake .. -DCMAKE_MODULE_PATH=$(pwd)/../../ZMusic/cmake -DZMusic_DIR=$(pwd)/../../ZMusic/build_install/lib/cmake/ZMusic

So in the classic CMake way you had to specify two variables (include dir and library). In the half baked way (with GZDoom's find module for ZMusic), you had to point it to your install and then modify ZMUSIC_LIBRARY to hack in the dependencies (two variables). In the new way you need to point to the ZMusic install and provide it the find modules for the dependencies, also two variables. It's two variables in all cases, so I'm failing to see an extra step here. Just different ones.

Remember, of course, that if you install ZMusic system wide and build in the standard dynamic configuration then everything gets picked up automatically with zero variables, and no need for a find module in GZDoom. Additionally the need to specify the location of ZMusic's find modules is only needed in the all static library case since the sndfile et al targets don't get depended upon if the DYN_ options are on. Oh and in that case it goes from two variables down to one since you don't need to worry about the ALSA dependency. ;)

I'm not going to say there aren't some cases where the modern style does make things slightly more obtuse, but personally I've found those to be the exception rather than the rule. Usually the common case becomes easier and the code becomes cleaner (not necessarily less code, just centralized).

[alexey-lysiuk]

I didn't encounter linker errors with ALSA. It could be a build without it, or I specified its library in CMAKE_EXE_LINKER_FLAGS variable as it is needed for all-static build (which is my primary interest).

My extra steps complain wasn't about a number of additional CMake arguments. It was about extra file(s) that should be obtained somehow. For now, I will keep dependency CMake modules along with prebuilt ZMusic static library.