Test agent execution on legacy systems

The agent compatibility requirements are:

Platform	Version
Red Hat (and similar)	≥ 7
Debian	≥ 10
Amazon Linux	≥ 2
Windows Server	≥ 2012 R2
Windows	≥ 10

However, we want to support C++ 20. So, it's desiderable to build the agent on Debian 11+ (or Ubuntu 22.04) and CentOS 8 or greater.

We need to test whether an agent executable built on a modern OS can run on a legacy platform. In addition, we should test whether a statically linked binary can run.

Test battery

Build platform	Run platform	Mode
Ubuntu 22.04	Debian 10	Dynamic linking
Ubuntu 22.04	Debian 10	Static linking
Rocky Linux 8	CentOS 7	Dynamic linking
Rocky Linux 8	CentOS 7	Static linking

Update

Got started with the issues, provisioning the necessary VMs.

Update

I've encountered several issues trying to build in Rocky Linux 8.

Two vagrant boxes were used: bento/rockylinux-8 and eurolinux-vagrant/rocky-8. The latter was used more extensively in the end.

Initially an attempt was made that resulted in compiling with missing symbols related to Lua, specifically to the lua_newuserdata symbol.

In Lua 5.4 and above, this function is no longer available. RPM (the library that requires Lua) specifies a minimum Lua version of 5.2. Yet, there's a compile definition LUA_COMPAT_5_3 that could fix this, but it wasn't explored. There's also a Vcpkg package https://vcpkg.io/en/package/lua-compat53 that could maybe serve as RPM's dependency.

Starting from scratch I tried to follow the packages/rpms/amd64/agent/Dockerfile used for creating packages. Still there were many problems, the repositories used for Centos 7 are not available to Rocky 8, and some of the packages listed do not seem to exist or have different names.

It is also worth mentioning that the Dockerfile for creating the package includes build steps for Perl, Lua, Sqlite, Popt, Rpm... some of which are dependencies that are already handled by Vcpkg. These Dockerfiles should be revised because they probably include steps required for 4.x and earlier, but not needed for 5.x.

After some trial and error I was able to build the Wazuh Agent on Rocky Linux 8, tests were able to run and they all passed.

Some changes were needed to build:

diff --git a/src/common/data_provider/CMakeLists.txt b/src/common/data_provider/CMakeLists.txt
index 61390269e..5fdb05857 100644
--- a/src/common/data_provider/CMakeLists.txt
+++ b/src/common/data_provider/CMakeLists.txt
@@ -191,6 +191,7 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
   find_library(LIBDB_LIBRARY NAMES db REQUIRED)
   find_library(LIBRPM_LIBRARY NAMES rpm REQUIRED)
   find_library(LIBRPMIO_LIBRARY NAMES rpmio REQUIRED)
+  find_package(unofficial-libmagic REQUIRED)
   target_link_libraries(sysinfo PUBLIC
     ${LIBDB_LIBRARY}
     ${PROC_NG_LIBRARY}
@@ -199,6 +200,7 @@ if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
     LibArchive::LibArchive
     ${POPT_LIBRARY}
     ${LUA_LIBRARIES}
+    unofficial::libmagic::libmagic
   )
 endif(CMAKE_SYSTEM_NAME STREQUAL "Linux")

diff --git a/src/ports-overlay/librpm/vcpkg.json b/src/ports-overlay/librpm/vcpkg.json
index aa78964ff..281a759c8 100644
--- a/src/ports-overlay/librpm/vcpkg.json
+++ b/src/ports-overlay/librpm/vcpkg.json
@@ -12,6 +12,15 @@
     {
       "name": "vcpkg-cmake-config",
       "host": true
+    },
+    {
+      "name": "libmagic"
+    },
+    {
+      "name": "sqlite3"
+    },
+    {
+      "name": "lua"
     }
   ]
 }
diff --git a/src/vcpkg.json b/src/vcpkg.json
index 4ad454e46..90f660f33 100644
--- a/src/vcpkg.json
+++ b/src/vcpkg.json
@@ -26,6 +26,10 @@
         "name": "curl",
         "version>=": "8.5.0"
       },
+      {
+        "name": "libmagic",
+        "version>=": "5.45"
+      },
       {
         "name": "fmt",
         "version>=": "10.2.1"

These changes highlight that magic.h was not possible to obtain without adding Libmagic as a dependency. Also, by adding the Libmagic, sqlite3 and lua dependencies to the RPM port, they are built before RPM, which conceptually is correct.

Update

To generate a static build I added the following command in src/CMakeLists.txt:

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -static-libgcc -static-libstdc++")

The static wazuh-agent binary built on Rocky 8 fails to run in CentOS 7:

[root@localhost vagrant]# uname -a
Linux localhost.localdomain 3.10.0-1160.95.1.el7.x86_64 #1 SMP Mon Jul 24 13:59:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost vagrant]# ldd wazuh-agent 
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agent)
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by ./wazuh-agent)
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agent)
        linux-vdso.so.1 =>  (0x00007ffe451d5000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f8428a2b000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f842880f000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f842850d000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f842813f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8428c2f000)
[root@localhost vagrant]# ./wazuh-agent 
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agent)
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by ./wazuh-agent)
./wazuh-agent: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agent)

The issue here is that wazuh-agent binary was built on a system with a glibc version that is newer than what is available on a CentOS 7 system. This results in the binary requiring GLIBC_2.18, GLIBC_2.25, and GLIBC_2.28, which CentOS 7 does not provide (it typically has GLIBC 2.17).

The binary is linked dynamically to libc.so.6 (the C standard library), and the version of glibc on Rocky 8 is newer than on the target system. This means the binary depends on features or symbols that exist in the newer version of glibc, which are not present in glibc 2.17.

It may be possible to build on Rocky 8 but link against an older glibc version that will be supported by CentOS 7.

Update

GLIBC required by the agent built on Rocky Linux 8 with GCC 13:

GLIBC_2.3
GLIBC_2.2.5
GLIBC_2.3.3
GLIBC_2.3.2
GLIBC_2.15
GLIBC_2.28
GLIBC_2.12
GLIBC_2.9
GLIBC_2.27
GLIBC_2.7
GLIBC_2.17
GLIBC_2.8
GLIBC_2.16
GLIBC_2.18
GLIBC_2.14
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.6
GLIBC_2.25

GLIBC required by wazuh agent built in CentOS 7 with GCC 13.

GLIBC_2.3
GLIBC_2.2.5
GLIBC_2.3.3
GLIBC_2.3.2
GLIBC_2.15
GLIBC_2.12
GLIBC_2.9
GLIBC_2.7
GLIBC_2.17
GLIBC_2.8
GLIBC_2.16
GLIBC_2.14
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.6

The binary built on CentOS 7 runs on CentOS 7 with GCC 13, CentOS 7 without GCC 13, and Rocky Linux 8.

Update

Linking libc statically is highly discouraged, see:

https://blogs.oracle.com/solaris/post/static-linking-where-did-it-go
- "If an application is built against libc.a, then any kernel interface it references is extracted from the archive and becomes a part of the application. Thus, this application can only run on a kernel that is in-sync with the kernel interfaces used. Should these interfaces change, the application is treading on shaky ground."
https://rockylinux.pkgs.org/8/rockylinux-devel-x86_64/glibc-static-2.28-251.el8_10.5.x86_64.rpm.html
- "The glibc-static package contains the C library static libraries for -static linking. You don't need these, unless you link statically, which is highly discouraged."

There are many more sources discouraging the use of a static libc library, these were just two.

Update

Using the -static flag raises many warnings, and the resulting binary crashes with a segmentation fault in the same system that was built:

/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/debug/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o): in function `dlfcn_load':                        [65/1820]
/home/vagrant/w2/src/vcpkg/buildtrees/openssl/x64-linux-dbg/../src/nssl-3.3.2-d50ad2b869.clean/crypto/dso/dso_dlfcn.c:116: warning: Using 'dlopen' in statically linked applications requires at runtime the shared
 libraries from the glibc version used for linking                                                                                                                                                                 
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/lib/libgtest.a(gtest-all.cc.o): in function `testing::internal::StreamingListener::SocketWriter::MakeConnect
ion()':                                                                                                                                                                                                            
gtest-all.cc:(.text+0xd9ac): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking                                       
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/debug/lib/libcrypto.a(libcrypto-lib-bio_sock.o): in function `BIO_gethostbyname':                           
/home/vagrant/w2/src/vcpkg/buildtrees/openssl/x64-linux-dbg/../src/nssl-3.3.2-d50ad2b869.clean/crypto/bio/bio_sock.c:126: warning: Using 'gethostbyname' in statically linked applications requires at runtime the 
shared libraries from the glibc version used for linking                                                                                                                                                           
[100%] Built target agent_registration_test                                                                                                                                                                        
[100%] Linking CXX executable message_queue_utils_test                                                                                                                                                             
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/debug/lib/libcrypto.a(libcrypto-lib-dso_dlfcn.o): in function `dlfcn_load':                                 
/home/vagrant/w2/src/vcpkg/buildtrees/openssl/x64-linux-dbg/../src/nssl-3.3.2-d50ad2b869.clean/crypto/dso/dso_dlfcn.c:116: warning: Using 'dlopen' in statically linked applications requires at runtime the shared
 libraries from the glibc version used for linking                                                                                                                                                                 
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/lib/libgtest.a(gtest-all.cc.o): in function `testing::internal::StreamingListener::SocketWriter::MakeConnect
ion()':                                                                                                                                                                                                            
gtest-all.cc:(.text+0xd9ac): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking                                       
/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: ../../vcpkg_installed/x64-linux/debug/lib/libcrypto.a(libcrypto-lib-bio_sock.o): in function `BIO_gethostbyname':                           
/home/vagrant/w2/src/vcpkg/buildtrees/openssl/x64-linux-dbg/../src/nssl-3.3.2-d50ad2b869.clean/crypto/bio/bio_sock.c:126: warning: Using 'gethostbyname' in statically linked applications requires at runtime the 
shared libraries from the glibc version used for linking                                                                                                                                                           
[100%] Built target message_queue_utils_test                                                                                                                                                                       
[root@localhost w2]# build/wazuh-agent --run                                                                                                                                                                       
[2024-11-08 13:11:26.895] [wazuh-agent] [info] [INFO] [process_options_unix.cpp:37] [StartAgent] Starting wazuh-agent                                                                                              
[2024-11-08 13:11:26.898] [wazuh-agent] [error] [ERROR] [configuration_parser.cpp:20] [ConfigurationParser] Using default values due to error parsing wazuh-agent.yml file: bad file: /etc/wazuh-agent/wazuh-agent.
yml                                                                                                                                                                                                                
[2024-11-08 13:11:26.899] [wazuh-agent] [info] [INFO] [inventory.cpp:16] [Start] Starting inventory.                                                                                                               
[2024-11-08 13:11:26.899] [wazuh-agent] [info] [INFO] [logcollector.cpp:17] [Start] Logcollector started                                                                                                           
[2024-11-08 13:11:26.900] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:787] [SyncLoop] Module started.                                                                                                            
[2024-11-08 13:11:26.900] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:771] [Scan] Starting evaluation.                                                                                                           
[2024-11-08 13:11:26.905] [wazuh-agent] [warning] [WARN] [communicator.cpp:54] [SendAuthenticationRequest] Failed to authenticate with the manager. Retrying in 30 seconds                                         
Segmentation fault (core dumped)

Update

- Ubuntu 22.04 - Debian 10 - Dynamic linking

When building the agent on Ubuntu 22.04 with dynamic linking and executing the binary on Debian 10, the following results were obtained:

./wazuh-agent 
    ./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./wazuh-agent)
    ./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./wazuh-agent)

It can be seen that the agent needs to load the following libraries:

ldd ./wazuh-agent 
./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.13' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./wazuh-agent)
    linux-vdso.so.1 (0x00007fff83c8a000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fb394819000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fb394695000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fb39467b000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb3944bb000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fb39582b000)

Debian 10 locally has an older version:

strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX
    GLIBCXX_3.4
    GLIBCXX_3.4.1
    GLIBCXX_3.4.2
    GLIBCXX_3.4.3
    GLIBCXX_3.4.4
    GLIBCXX_3.4.5
    GLIBCXX_3.4.6
    GLIBCXX_3.4.7
    GLIBCXX_3.4.8
    GLIBCXX_3.4.9
    GLIBCXX_3.4.10
    GLIBCXX_3.4.11
    GLIBCXX_3.4.12
    GLIBCXX_3.4.13
    GLIBCXX_3.4.14
    GLIBCXX_3.4.15
    GLIBCXX_3.4.16
    GLIBCXX_3.4.17
    GLIBCXX_3.4.18
    GLIBCXX_3.4.19
    GLIBCXX_3.4.20
    GLIBCXX_3.4.21
    GLIBCXX_3.4.22
    GLIBCXX_3.4.23
    GLIBCXX_3.4.24
    GLIBCXX_3.4.25
    GLIBC_2.2.5
    GLIBC_2.3
    GLIBC_2.14
    GLIBC_2.18
    GLIBC_2.16
    GLIBC_2.17
    GLIBC_2.3.2

Update

A dynamic build with gcc13 on Rocky Linux 8 fails as expected on CentOS 7 without gcc13:

[vagrant@localhost ~]$ ./wazuh-agentd
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agentd)
[vagrant@localhost ~]$ ldd wazuh-agentd
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agentd)
        linux-vdso.so.1 =>  (0x00007ffdbada2000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f974c0dd000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f974bed9000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f974bcbd000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f974b9b5000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f974b79f000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f974b3d1000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f974c3df000)
[vagrant@localhost ~]$ uname -a
Linux localhost.localdomain 3.10.0-1160.95.1.el7.x86_64 #1 SMP Mon Jul 24 13:59:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

And it also fails in CentOS 7 with gcc13:

[root@localhost vagrant]# ./wazuh-agentd
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agentd)
[root@localhost vagrant]# ldd wazuh-agentd
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.28' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by ./wazuh-agentd)
./wazuh-agentd: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by ./wazuh-agentd)
        linux-vdso.so.1 =>  (0x00007fff83d24000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f5633be6000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f56339e2000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f56337c6000)
        libstdc++.so.6 => /usr/local/gcc-13.2.0/lib64/libstdc++.so.6 (0x00007f5633374000)
        libgcc_s.so.1 => /usr/local/gcc-13.2.0/lib64/libgcc_s.so.1 (0x00007f5633151000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f5632d83000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f5633ee8000)
[root@localhost vagrant]# uname -a
Linux localhost.localdomain 3.10.0-1160.95.1.el7.x86_64 #1 SMP Mon Jul 24 13:59:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Update

- Ubuntu 22.04 - Debian 10 - Static linking

To build the agent on Ubuntu 22.04 with static linking it was necessary to add the following line in src/CMakelists.txt: set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static-libgcc -static-libstdc++")

When executing the binary on Debian 10, the following results were obtained:

./wazuh-agent 
./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./wazuh-agent)

It can be seen that the agent needs to load the following libraries:

ldd ./wazuh-agent 
./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./wazuh-agent)
    linux-vdso.so.1 (0x00007ffc9e53e000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3af1a21000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3af1861000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f3af2b4e000)

Update

Debian 10 - Static linking

To build the agent on Debian 10 with static linking (using g++-13.2.0) it was necessary to add the following line in src/CMakelists.txt: set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static-libgcc -static-libstdc++")

When executing the binary on Debian 10 (clean) , the following results were obtained:

./wazuh-agent 
./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libpthread.so.0: version `GLIBC_2.30' not found (required by ./wazuh-agent)

It can be seen that the agent needs to load the following libraries:

ldd ./wazuh-agent 
./wazuh-agent: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./wazuh-agent)
./wazuh-agent: /lib/x86_64-linux-gnu/libpthread.so.0: version `GLIBC_2.30' not found (required by ./wazuh-agent)
    linux-vdso.so.1 (0x00007ffedd8ee000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f43bdcf2000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f43bdced000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f43bdccc000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f43bdb0c000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f43bde80000)

Update

By default the vagrant box debian/buster64 includes this as GLIBC:

root@buster:/home/vagrant# grep -a -o '[[:print:]]\{4,\}' /lib/x86_64-linux-gnu/libc.so.6 | grep GLIB
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_2.13
GLIBC_2.14
GLIBC_2.15
GLIBC_2.16
GLIBC_2.17
GLIBC_2.18
GLIBC_2.22
GLIBC_2.23
GLIBC_2.24
GLIBC_2.25
GLIBC_2.26
GLIBC_2.27
GLIBC_2.28
GLIBC_PRIVATE
GNU C Library (Debian GLIBC 2.28-10+deb10u2) stable release version 2.28.

After following the steps from the Dockerfile this is the result:

root@buster:/home/vagrant/gcc-13.2.0# strings /usr/lib/x86_64-linux-gnu/libc.so.6 | grep GLIBC
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_2.13
GLIBC_2.14
GLIBC_2.15
GLIBC_2.16
GLIBC_2.17
GLIBC_2.18
GLIBC_2.22
GLIBC_2.23
GLIBC_2.24
GLIBC_2.25
GLIBC_2.26
GLIBC_2.27
GLIBC_2.28
GLIBC_2.29
GLIBC_2.30
GLIBC_PRIVATE
GNU C Library (Debian GLIBC 2.31-13+deb11u11) stable release version 2.31.

Some of the packages being installed upgrade the GNU C Library.

Starting from scratch and only installing the minimum required packages. Output of history:

   10  apt install g++-8
   11  apt install gcc-8
   12  update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 80
   13  update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 80
   14  apt install make
   17  apt install --only-upgrade binutils
   24  export CPLUS_INCLUDE_PATH="/usr/local/gcc-13.2.0/include/c++/13.2.0/"
   25  export LD_LIBRARY_PATH="/usr/local/gcc-13.2.0/lib64/"
   27  export PATH="/usr/local/gcc-13.2.0/bin:${PATH}"
   28  apt install pkg-config
   29  export PKG_CONFIG_PATH="/usr/local/lib/pkgconfig:${PKG_CONFIG_PATH}"
   30  curl -OL https://github.com/Kitware/CMake/releases/download/v3.30.3/cmake-3.30.3.tar.gz &&     tar -zxf cmake-3.*.tar.gz && cd cmake-3.* && ./bootstrap &&     make -j$(nproc) && make install && ln -fs /usr/local/bin/cmake /usr/bin/cmake &&     cd / && rm -rf cmake-*
   35  apt install libssl-dev
   36  ./bootstrap &&     make -j$(nproc) && make install && ln -fs /usr/local/bin/cmake /usr/bin/cmake &&     cd / && rm -rf cmake-*
   38  curl -sO https://lua.org/ftp/lua-5.4.7.tar.gz && tar -xzvf lua-5.4.7.tar.gz &&     cd lua-5.4.7 && make -j$(nproc) linux CFLAGS+="-fPIC" LDFLAGS+="-fPIC" && make install &&     ln -fs /usr/local/bin/lua /usr/bin/lua && cd / && rm -rf lua*
   39  mkdir -p /usr/local/lib/pkgconfig &&     echo "# Package Information for pkg-config" > /usr/local/lib/pkgconfig/lua.pc &&     echo "prefix=/usr/local" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "exec_prefix=\${prefix}" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "libdir=\${exec_prefix}/lib" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "includedir=\${prefix}/include" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "Name: Lua" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "Description: Lua" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "Version: 5.4.7" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "Libs: -L\${libdir} -llua -lm" >> /usr/local/lib/pkgconfig/lua.pc &&     echo "Cflags: -I\${includedir}" >> /usr/local/lib/pkgconfig/lua.pc
   41  apt install git
   42  git clone https://github.com/wazuh/wazuh-agent
   44  cd wazuh-agent/
   46  git submodule update --init --recursive
   48  apt install ninja
   49  apt install ninja-build
   55  sudo apt-get install curl zip unzip tar
   57  apt install autoconf
   60  apt install autopoint
   64  apt install libtool
   68  apt install libmagic-dev
   71  apt install libpopt-dev
   74  apt install libsqlite3-dev
   80  apt install gettext
   83  apt install libbz2-dev
   87  apt install libncurses5-dev libncursesw5-dev
   88  rm -rf build && cmake src -B build -DBUILD_TESTS=1
   89  cmake --build build -j8

Contents of glibc after compiling wazuh-agent:

root@buster:/wazuh-agent# grep -a -o '[[:print:]]\{4,\}' /lib/x86_64-linux-gnu/libc.so.6 | grep GLIB
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_2.13
GLIBC_2.14
GLIBC_2.15
GLIBC_2.16
GLIBC_2.17
GLIBC_2.18
GLIBC_2.22
GLIBC_2.23
GLIBC_2.24
GLIBC_2.25
GLIBC_2.26
GLIBC_2.27
GLIBC_2.28
GLIBC_PRIVATE
GNU C Library (Debian GLIBC 2.28-10+deb10u4) stable release version 2.28.

Binary links:

root@buster:/wazuh-agent# uname -a && ldd build/wazuh-agent
Linux buster 4.19.0-25-amd64 #1 SMP Debian 4.19.289-2 (2023-08-08) x86_64 GNU/Linux
    linux-vdso.so.1 (0x00007ffe255f4000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f962046b000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9620466000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9620445000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9620285000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f96205f7000)

Running on a clean Debian 10:

vagrant@buster:~$ uname -a && ldd wazuh-agent && ./wazuh-agent --run
Linux buster 4.19.0-25-amd64 #1 SMP Debian 4.19.289-2 (2023-08-08) x86_64 GNU/Linux
        linux-vdso.so.1 (0x00007ffde879d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd9fcd60000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd9fcd5b000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd9fcd3a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd9fcb7a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fd9fceeb000)
[2024-11-11 19:01:43.741] [wazuh-agent] [info] [INFO] [process_options_unix.cpp:37] [StartAgent] Starting wazuh-agent
[2024-11-11 19:01:43.742] [wazuh-agent] [error] [ERROR] [configuration_parser.cpp:52] [ConfigurationParser] Using default values due to error parsing wazuh-agent.yml file: bad file: /etc/wazuh-agent/wazuh-agent.yml
[2024-11-11 19:01:43.743] [wazuh-agent] [info] [INFO] [inventory.cpp:16] [Start] Starting inventory.
[2024-11-11 19:01:43.743] [wazuh-agent] [info] [INFO] [logcollector.cpp:17] [Start] Logcollector started
[2024-11-11 19:01:43.744] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:787] [SyncLoop] Module started.
[2024-11-11 19:01:43.745] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:771] [Scan] Starting evaluation.
[2024-11-11 19:01:43.749] [wazuh-agent] [warning] [WARN] [communicator.cpp:66] [SendAuthenticationRequest] Failed to authenticate with the manager. Retrying in 30 seconds
[2024-11-11 19:01:43.750] [wazuh-agent] [warning] [WARN] [http_client.cpp:99] [Co_PerformHttpRequest] Failed to send http request. /api/v1/events/stateful. Retrying in 30 seconds
[2024-11-11 19:01:43.750] [wazuh-agent] [warning] [WARN] [http_client.cpp:99] [Co_PerformHttpRequest] Failed to send http request. /api/v1/events/stateless. Retrying in 30 seconds
[2024-11-11 19:01:43.750] [wazuh-agent] [warning] [WARN] [http_client.cpp:99] [Co_PerformHttpRequest] Failed to send http request. /api/v1/commands. Retrying in 30 seconds
[2024-11-11 19:01:43.789] [wazuh-agent] [info] [INFO] [inventoryImp.cpp:782] [Scan] Evaluation finished.

Conclusion

I was able to build the new wazuh-agent on CentOS 7 and Debian 10 statically (static libgcc and libstdc++) and run it on other clean provisioned systems. It's not possible to statically link glibc. It could be worth it to investigate if it is possible to build on newer systems older versions of glibc to link against, though the process seems to be more involved/complicated.

Update

Looking into crosstool-ng as a possible way to create toolchains that can cross compile for older OSs.

Update

Running tests with crosstool-ng. With the tool ct-ng menuconfig I tried to create a toolchain for cross compiling with the following options: glibc 2.17, linux kernel 3.10, x86-64. Then proceeded to compile wazuh which failed at the vcpkg configuration stage due to an error with autoconf. Autoconf can be added to the toolchain, which will be the following test to perform after looking more deeply into the current issue, since there may be another workaround.

Update

Some changes were needed to build the agent.

RPM dependency's port overlay needs to declare its own dependencies (Lua, libmagic, libpopt, sqlite3, bzip2) so they are built before RPM and used. This makes me wonder if the current port overlay is using the vcpkg downloaded libs or the system wide available ones.
RPM's configure step requires Lua to be installed with pkg-config, I wasn't able to achieve this so I had to create a lua.pc file and place it manually in the expected folder.
When building libpopt for debug builds, the library is called poptd, it seems that despite the build type, RPM is always built as Release, and expects its dependencies to be release builds as well.
It wasn't enough to set CMAKE_C_FLAGS and CMAKE_CXX_FLAGS to make the build static, unlike previous tests, I had to also set CMAKE_EXE_LINKER_FLAGS
Once the object was built and tested on a different folder or system it runs without crashing but there's an error logged:

error: Unable to open /home/vagrant/wazuh-agent/build/vcpkg_installed/x64-linux/debug/lib/rpm/rpmrc for reading: No such file or directory.

Apparently this error is also present when building without the toolchain, and without static flags.

Update

Looking into error:

error: Unable to open /home/vagrant/wazuh-agent/build/vcpkg_installed/x64-linux/debug/lib/rpm/rpmrc for reading: No such file or directory.

No significant progress, the error is in master as well and also happens when building dynamically.

Update

It is possible to install devtoolset-10-gcc and devtoolset-10-gcc-g++ with the following list of repositories on CentOS 7:

[base]
name=CentOS-$releasever - Base
baseurl=https://vault.centos.org/centos/7/os/x86_64/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=https://vault.centos.org/centos/7/updates/x86_64/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras
baseurl=https://vault.centos.org/centos/7/extras/x86_64/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus
baseurl=https://vault.centos.org/centos/7/centosplus/x86_64/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that extend functionality of existing packages
[scl]
name=CentOS-$releasever - SCL
baseurl=https://archive.kernel.org/centos-vault/centos/7/sclo/x86_64/rh/
gpgcheck=0
enabled=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

The CMakeLists.txt needs to be modified with

add_compile_options(-fcoroutines)                                                                        │
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fcoroutines -static-libgcc -static-libstdc++")                  │
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static-libgcc -static-libstdc++")

Since this a differente compiler version, different warnings are triggered and require attention, but other than that and the fcoroutines flag, the agent is correctly built without errors or warnings.

It runs on CentOS7, Debian10 and tested on the host machine with a Ubuntu22.

Update

The build using crosstool-ng also worked despite the rpmrc error mentioned in earlier comments (the same error is found in master when building dynamically and in the host machine with the native compiler, and does not crash the application).

The .config file used for creating the cross compiler toolchain is attached here as config.txt. In a nutshell, glibc 2.17, linux kernel 3.10 are the selected options. Together with c++ in the C compiler. The rest of the process is pretty similar and could be packed in a dockerfile.

config.txt

wazuh / wazuh-agent