Can't build for Amazon Linux

[ffxsam]

I'm trying to build this to run within an AWS Lambda function. I can't get past the CMake step:

sh-4.2# cmake3 -DWIDGETS_GUI=OFF -DQUICK_GUI=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/tmp ../src/subdirs/tageditor

-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for iconv
-- Looking for iconv - found
-- Using iconv from the standard library for target c++utilities.
-- Using __gnu_cxx::stdio_filebuf for NativeFileStream
-- Performing Test STD_FILESYSTEM_TEST_0
-- Performing Test STD_FILESYSTEM_TEST_0 - Failed
-- Performing Test STD_FILESYSTEM_TEST_1
-- Performing Test STD_FILESYSTEM_TEST_1 - Failed
-- Performing Test STD_FILESYSTEM_TEST_2
-- Performing Test STD_FILESYSTEM_TEST_2 - Failed
CMake Error at /work/src/c++utilities/cmake/modules/3rdParty.cmake:369 (message):
  Unable to compile a simple std::filesystem example.  A compiler supporting
  C++17 is required to build this project.
Call Stack (most recent call first):
  /work/src/c++utilities/CMakeLists.txt:171 (use_standard_filesystem)

-- Configuring incomplete, errors occurred!
See also "/work/build/CMakeFiles/CMakeOutput.log".
See also "/work/build/CMakeFiles/CMakeError.log".

I followed the instructions as far as I can tell. Any help would be greatly appreciated!

[Martchus]

The check for the availability of std::filesystem fails (as the error message tells you). Maybe this C++ feature is indeed not available in your build environment. Then you need to upgrade your environment. The check might also fail because there's some bigger problem with the build environment. Without knowing the contents of the log files mentioned at the bottom of your paste I can only guess what the problem is.

[Martchus]

Note that your log says "GNU 7.3.1". I would really recommend upgrading to a newer version. GCC 9.1 was the first release with non-experimental C++17 support and C++17 is required to build the latest version of Tag Editor and its dependencies.

[ffxsam]

Aha, progress! I installed gcc10-c++ and that worked.

$ cmake3 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/work -DWIDGETS_GUI=OFF -DQUICK_GUI=OFF -DCMAKE_CXX_COMPILER=/usr/bin/gcc10-g++ -DCMAKE_C_COMPILER=/usr/bin/gcc10-cc ../src/subdirs/tageditor

I had to specify both the C++ compiler as well as the C compiler for some reason. But then that works, and the make produces:

[  3%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/chrono/datetime.cpp.o
[  3%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/application/fakeqtconfigarguments.cpp.o
[  3%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/application/argumentparser.cpp.o
[  4%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/application/commandlineutils.cpp.o
[  5%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/chrono/timespan.cpp.o
[  6%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/chrono/period.cpp.o
[  7%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/conversion/conversionexception.cpp.o
[  8%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/conversion/stringconversion.cpp.o
[  9%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/ansiescapecodes.cpp.o
[ 10%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/binaryreader.cpp.o
[ 11%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/binarywriter.cpp.o
[ 12%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/bitreader.cpp.o
[ 13%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/buffersearch.cpp.o
[ 14%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/inifile.cpp.o
[ 15%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/path.cpp.o
[ 16%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/nativefilestream.cpp.o
[ 17%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/io/misc.cpp.o
[ 18%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/misc/math.cpp.o
[ 19%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/misc/parseerror.cpp.o
[ 20%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/misc/levenshtein.cpp.o
[ 21%] Building CXX object c++utilities/CMakeFiles/c++utilities.dir/tests/testutils.cpp.o
[ 22%] Linking CXX static library libc++utilities.a
Error running link command: No such file or directory
make[2]: *** [c++utilities/libc++utilities.a] Error 2
make[1]: *** [c++utilities/CMakeFiles/c++utilities.dir/all] Error 2
make: *** [all] Error 2

I'm not a C/C++ guy these days, so I'm not sure what link command it's looking for, specifically. The Amazon Linux Docker image is pretty barebones so I had to install all the development tools, though clearly I missed some!

[Martchus]

For linking static libraries ar is supposed to be used. Maybe it is not there or maybe CMake passes a non-existing file to it. You can add -DCMAKE_VERBOSE_MAKEFILE=ON to the CMake arguments for more verbose output.

I don't know the distribution you're using but it could be that gcc10-c++ really only covers the compiler and the rest of the build tools needs to be installed explicitly via some binutils package.

[ffxsam]

Thanks for your help! I'll try to see if I can install ar, and I'll try the verbose setting too.

For reference, I'm using the amazonlinux:latest Docker image to build this.

[ffxsam]

Here's the verbose output:

/usr/bin/cmake3 -S/work/src/subdirs/tageditor -B/work/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake3 -E cmake_progress_start /work/build/CMakeFiles /work/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/work/build'
make -f c++utilities/CMakeFiles/c++utilities.dir/build.make c++utilities/CMakeFiles/c++utilities.dir/depend
make[2]: Entering directory `/work/build'
cd /work/build && /usr/bin/cmake3 -E cmake_depends "Unix Makefiles" /work/src/subdirs/tageditor /work/src/c++utilities /work/build /work/build/c++utilities /work/build/c++utilities/CMakeFiles/c++utilities.dir/DependInfo.cmake --color=
make[2]: Leaving directory `/work/build'
make -f c++utilities/CMakeFiles/c++utilities.dir/build.make c++utilities/CMakeFiles/c++utilities.dir/build
make[2]: Entering directory `/work/build'
[  1%] Linking CXX static library libc++utilities.a
cd /work/build/c++utilities && /usr/bin/cmake3 -P CMakeFiles/c++utilities.dir/cmake_clean_target.cmake
cd /work/build/c++utilities && /usr/bin/cmake3 -E cmake_link_script CMakeFiles/c++utilities.dir/link.txt --verbose=1
CMAKE_AR-NOTFOUND qc libc++utilities.a  CMakeFiles/c++utilities.dir/application/argumentparser.cpp.o CMakeFiles/c++utilities.dir/application/commandlineutils.cpp.o CMakeFiles/c++utilities.dir/application/fakeqtconfigarguments.cpp.o CMakeFiles/c++utilities.dir/chrono/datetime.cpp.o CMakeFiles/c++utilities.dir/chrono/period.cpp.o CMakeFiles/c++utilities.dir/chrono/timespan.cpp.o CMakeFiles/c++utilities.dir/conversion/conversionexception.cpp.o CMakeFiles/c++utilities.dir/conversion/stringconversion.cpp.o CMakeFiles/c++utilities.dir/io/ansiescapecodes.cpp.o CMakeFiles/c++utilities.dir/io/binaryreader.cpp.o CMakeFiles/c++utilities.dir/io/binarywriter.cpp.o CMakeFiles/c++utilities.dir/io/bitreader.cpp.o CMakeFiles/c++utilities.dir/io/buffersearch.cpp.o CMakeFiles/c++utilities.dir/io/inifile.cpp.o CMakeFiles/c++utilities.dir/io/path.cpp.o CMakeFiles/c++utilities.dir/io/nativefilestream.cpp.o CMakeFiles/c++utilities.dir/io/misc.cpp.o CMakeFiles/c++utilities.dir/misc/math.cpp.o CMakeFiles/c++utilities.dir/misc/parseerror.cpp.o CMakeFiles/c++utilities.dir/misc/levenshtein.cpp.o CMakeFiles/c++utilities.dir/tests/testutils.cpp.o
Error running link command: No such file or directory
make[2]: *** [c++utilities/libc++utilities.a] Error 2
make[2]: Leaving directory `/work/build'
make[1]: *** [c++utilities/CMakeFiles/c++utilities.dir/all] Error 2
make[1]: Leaving directory `/work/build'
make: *** [all] Error 2

I tried -DCMAKE_AR=/usr/bin/ar but that didn't help (and ar is now installed).

[Martchus]

Setting CMAKE_AR via CMake arguments should have worked (if it exists at the specified path). Maybe it helps to start again using a clean build directory (if you haven't already tried that). Note that my build scripts do nothing special in that regard. This looks like a general problem of your build environment (ar is a very essential tool and should be generally available in a GNU/Linux C/C++ build environment).

You could alternatively build shared libraries by adding -DBUILD_SHARED_LIBS=ON to the CMake arguments. Then ar shouldn't be required. (When targeting GNU/Linux that might be the preferred option anyways.)

By the way, I normally use a CMake toolchain file to set a different compiler or simply set environment variables, e.g.

export CC=gcc-8
export CXX=g++-8

Maybe these approaches work better than trying to specify compiler paths via CMake arguments. (Setting environment variables like this definitely works. I'm doing the same for the AppImage build where I also need to a newer GCC version than the default one, see https://build.opensuse.org/package/view_file/home:mkittler:appimage/tageditor/appimage.yml?expand=1.)

[ffxsam]

Static libraries are when the libraries are built into the executable, correct? That's what I'll need in this case. The AWS Lambda environment is super barebones and there can't be any external dependencies.

I'll give it another shot. That AppImage link helps! I can probably set up something similar in the Docker config file and build script. Thanks!

[Martchus]

Static libraries are when the libraries are built into the executable, correct?

Yes and no. With -DBUILD_SHARED_LIBS=OFF (the default) all libraries produced by the build will be static libraries and the final executable will be linked against such static libraries and hence not depend on them at runtime. However, there are further external dependencies such as zlib, the C++ standard library and glibc. These libraries are not produced by the build but are simply located within the build environment. By default CMake prefers shared libraries when searching the build environment for such dependencies and my build scripts do not change the default by default. You could try to change that by adding -DSTATIC_LINKAGE=ON to the CMake arguments but I would not recommend it when targeting GNU/Linux (the setting is meant for targeting mingw-w64/Windows where static "system libraries" are actually just import libraries and thus lead to dynamic linkage against system DLLs after all).

[ffxsam]

Built! Although after all that, I realize it doesn't tag WAV or AIFF files. I suppose I should've checked into that first! 🤦‍♂️

Thanks for walking me through it, though.

[Martchus]

I've never implemented these. Normally FLAC does a better job for storing lossless audio data so I never had the use case. But glad to here you could build it.