Description

Hello team,

After some open issues due to problems installing the Wazuh agent on the AIX operating system.

We have detected that it is necessary to move the libatomic.a to a specific directory /usr/lib, or at least modify the LIBPATH environment variable to contain the path where this library is located, by default /opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8/ppc64/.

Error installing wazuh-agent:

bash-5.1# WAZUH_MANAGER="10.0.0.2" rpm -ivh wazuh-agent-4.8.1-1.aix.ppc.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:wazuh-agent-4.8.1-1              ################################# [100%]
bash-5.1# /var/ossec/bin/wazuh-control start
exec(): 0509-036 Cannot load program /var/ossec/bin/wazuh-execd because of the following errors:
        0509-130 Symbol resolution failed for /var/ossec/lib/libwazuhext.so because:
        0509-136   Symbol _GLOBAL__AIXI_libatomic_so (number 167) is not exported from
                   dependent module /usr/lib/libatomic.a(libatomic.so.1).
        0509-136   Symbol _GLOBAL__AIXD_libatomic_so (number 168) is not exported from
                   dependent module /usr/lib/libatomic.a(libatomic.so.1).
        0509-136   Symbol __atomic_is_lock_free (number 170) is not exported from
                   dependent module /usr/lib/libatomic.a(libatomic.so.1).
        0509-192 Examine .loader section symbols with the
                 'dump -Tv' command.
wazuh-execd: Configuration error. Exiting

After moving libatomic.a, it works:

bash-5.1# cp /opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8/ppc64/libatomic.a /usr/lib
bash-5.1# /var/ossec/bin/wazuh-control start
Starting Wazuh v4.8.1...
Started wazuh-execd...
Started wazuh-agentd...
Started wazuh-syscheckd...
Started wazuh-logcollector...
Started wazuh-modulesd...
Completed.

This problem appears in AIX 7.X versions, since for version 6 the agent was correctly tested here:

https://github.com/wazuh/wazuh/issues/23963

We have managed to make it work with the workaround given in the mentioned issues, however, we want to investigate further to see if we can or should do something in the agent to make it work directly.

Caveats

Has the LIBPATH changed, or is it the location of the libraries?
Is there a guarantee that the libatomic.so file is always present on AIX 7.x?

Fix proposal

I think we can simply update LIBPATH at ossec-hids-aix.init.

Investigation

Upon further investigation, I have not found that we have modified the LIBPATH variable. Actually the problem comes because we are looking for it in the /usr/lib directory:

ifeq (${uname_S},AIX)
        DEFINES+=-DAIX -D__unix
        DEFINES+=-DHIGHFIRST
        OSSEC_CFLAGS+=-pthread
        OSSEC_LDFLAGS+=-pthread -L./lib
ifeq ($(INSTALLDIR),)
    INSTALLDIR = /var/ossec
endif
        CMAKE_OPTS+=-DINSTALL_PREFIX=${INSTALLDIR}
        OSSEC_LDFLAGS+='-Wl,-blibpath:${INSTALLDIR}/lib:/usr/lib:/lib'
        AR_LDFLAGS+=-pthread
        AR_LDFLAGS+='-Wl,-blibpath:${INSTALLDIR}/lib:/usr/lib:/lib'
        PATH:=${PATH}:/usr/vac/bin
        CC=gcc
        PRECOMPILED_OS:=aix

And yet for AIX 7.X versions, it seems that it is no longer found in that directory. I have found in an IBM docu, that for the Open XL C/C++ 17.1.1 compiler version, the libatomic library is no longer linked implicitly, and maybe this has also caused that it has been removed from /usr/lib: https://www.ibm.com/docs/en/SSRZSMX_17.1.1/pdf/migrate.pdf

In IBM Open XL C/C++ for AIX 17.1.1, many libraries are no longer linked implicitly, such as libatomic
and libm. If operations related to these libraries are used in your program, specify the corresponding
linking options explicitly, such as -latomic and -lm.

The library is present in the installation of gcc 8 itself, so a possible option would be as @vikman90 mentions to include it manually in the LIBPATH. It can also be done directly in the Makefile by adding the path:

$(OSSEC_SHARED) $(OSSEC_CFLAGS) libwazuhext/*.o -o $@ -static-libgcc -Wl,-blibpath:/opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8/ppc64:/usr/lib:/lib -latomic

@jotacarma90 thanks!

Action items

[ ] Check the library search order (in load time): According to the ld.so manual, the Linux loader first looks at the DT_RPATH section of the binary, and it later searches in LD_LIBRARY_PATH. We need to check if AIX works in the same way, either finding some documentation or checking it manually.
[ ] Implement the Makefile fix: Check if we can add some kind of wildcards to -blibpath and extend the flag with /opt/freeware/lib/gcc/powerpc-ibm-aix7.2.0.0/8/ppc64 (or similar).

Update

(26/08/2024) Requested VM to develop and test the solution. Providing the VM with the required dependencies to build and test the solution. (27/08/2024) Performed different tests with AIX 6 and AIX 7. Researching lib load procedure in both version, building different packages with an initial solution. Still some tests to perform. (27/08/2024) After differents test we have come up with the following conclusions:

In AIX 6 there is no GCC package installed by default, in our case, the package is installed when the machine is provisioned with the wazuh-packages script. And checking the installed packages we find:

# rpm -q --filesbypkg libgcc-6.3.0-1 | grep "libatomic*"
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libatomic.a
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libatomic.la
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libatomic.a
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libatomic.la
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libatomic.a
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libatomic.la
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libatomic.a
libgcc                    /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libatomic.la
libgcc                    /opt/freeware/lib/libatomic.a
libgcc                    /opt/freeware/lib/pthread/libatomic.a
libgcc                    /opt/freeware/lib/pthread/ppc64/libatomic.a
libgcc                    /opt/freeware/lib64/libatomic.a

# ls  -la /opt/freeware/lib/libatomic.a
lrwxrwxrwx    1 root     system           44 Aug 27 07:26 /opt/freeware/lib/libatomic.a -> gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libatomic.a

The libatomic.alibrary is installed with the libgcc-6.3.0-1 package, and checking /opt/freeware/lib/libatomic.a it's observer it's a symbolic link to the /opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libatomic.a file so we can simply add the /opt/freeware/lib path to link this library and solve the reported problem as done in https://github.com/wazuh/wazuh/commit/3e90c220f782def1fc8c4d6aa9047c8b1c69c06e.

In AIX 7 by default the GCC 8 version is installed, so when the machine is provisioned the following message appears:

# rpm -Uvh --nodeps http://packages-dev.wazuh.com/deps/aix/libgcc-6.3.0-1.aix7.2.ppc.rpm
Retrieving http://packages-dev.wazuh.com/deps/aix/libgcc-6.3.0-1.aix7.2.ppc.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
        package libgcc-1:8-1.ppc (which is newer than libgcc-6.3.0-1.ppc) is already installed

Performing the previous test we find the installed version is similar to AIX 6:

# rpm -q --filesbypkg libgcc-8-1.ppc
libgcc                    /opt/freeware/lib/libatomic.a
libgcc                    /opt/freeware/lib/libgcc_s.a
libgcc                    /opt/freeware/lib/pthread/libatomic.a
libgcc                    /opt/freeware/lib/pthread/libgcc_s.a
libgcc                    /opt/freeware/lib/pthread/ppc64/libatomic.a
libgcc                    /opt/freeware/lib/pthread/ppc64/libgcc_s.a
libgcc                    /opt/freeware/lib64/libatomic.a
libgcc                    /opt/freeware/lib64/libgcc_s.a
# ls -la /opt/freeware/lib/libatomic.a
lrwxrwxrwx    1 root     system           40 May 19 2022  /opt/freeware/lib/libatomic.a -> gcc/powerpc-ibm-aix7.2.0.0/8/libatomic.a

So we can also link to the same file as mentioned previously.

It's worth mentioning that IBM advises to move the libatomic.a file to /usr/lib folder when installing it, as seen in this documentation page so the reported behavior is expected if the library is not fully installed as required by IBM.

The proposed solution makes the compiler link with the /opt/freeware/lib letting the package find the library and making the agent start as expected.

This package wazuh-agent-4.9.1-1.aix6.1.ppc.zip has been built using the fix/25263-errors-related-to-libatomica-library-on-aix-7x branch which implements the proposed fix. It works for both 6 and 7 AIX versions:

# uname -a
AIX soaxp181 1 6 00CADA644C00

# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.9.1"
WAZUH_REVISION="40910"
WAZUH_TYPE="agent"

# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

# uname -a
AIX soaix426 2 7 00F9D80F4C00

# /var/ossec/bin/wazuh-control info
WAZUH_VERSION="v4.9.1"
WAZUH_REVISION="40910"
WAZUH_TYPE="agent"

# /var/ossec/bin/wazuh-control status
wazuh-modulesd is running...
wazuh-logcollector is running...
wazuh-syscheckd is running...
wazuh-agentd is running...
wazuh-execd is running...

wazuh / wazuh

Errors related to 'libatomic.a' library on AIX 7.X #25263