wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
9.44k stars 1.49k forks source link

Compilation on AIX #142

Closed jamesspi closed 5 years ago

jamesspi commented 6 years ago

Hi,

I'm looking to compile this on AIX, but I'm getting this error on compilation:

make: *** [client-agent/notify.o] Error 1

Looks like it's related to inotify - but this doesn't seem to be available for AIX.

Any suggestions please?

Thanks, James

vikman90 commented 6 years ago

Hi @jamesspi,

I think that your problem is not related to inotify: that file collects the agent information, adds some random data and sends it to the manager, so the issue may be related to any of these functions.

Please send us the error that the compiler reports about the problem, maybe it includes the line of code when the problem happens.

Thanks for your feedback. Best.

jamesspi commented 6 years ago

Hi @vikman90 ,

Thanks for the prompt response.

That is the only error I get unfortunately. Nothing else is reported.

Thanks, James

rnbwkat commented 6 years ago

Hi - I might be able to help. I have built OSSEC several dozen times on AIX and several versions, ranging from 5.x to 7.x. Can you provide the compile output from the make through the error?

jamesspi commented 6 years ago

Hi @rnbwkat @vikman90 , here is the output:

Para instalação em português, escolha [br]. 要使用中文进行安装, 请选择 [cn]. Für eine deutsche Installation, wählen Sie [de]. Για εγκατάσταση στα Ελληνικά, επιλέξτε [el]. For installation in English, choose [en]. Para instalar en español, elija [es]. Pour une installation en français, choisissez [fr] A Magyar nyelvű telepítéshez válassza [hu]. Per l'installazione in Italiano, scegli [it]. 日本語でインストールします.選択して下さい.[jp]. Voor installatie in het Nederlands, kies [nl]. Aby instalować w języku Polskim, wybierz [pl]. Для инструкций по установке на русском ,введите [ru]. Za instalaciju na srpskom, izaberi [sr]. ** Türkçe kurulum için seçin [tr]. (en/br/cn/de/el/es/fr/hu/it/jp/nl/pl/ru/sr/tr) [en]: Wazuh v2.0.1 Installation Script - http://www.wazuh.com

You are about to start the installation process of Wazuh. You must have a C compiler pre-installed in your system.

1- What kind of installation do you want (server, agent, local, hybrid or help)? agent

2- Setting up the installation environment.

3- Configuring Wazuh.

3.1- What's the IP Address or hostname of the Wazuh server?: xxxxxxxxxxxx

4- Installing the system

rnbwkat commented 6 years ago

Last question - gcc or native cc?

vikman90 commented 6 years ago

Hi @jamesspi @rnbwkat,

The error seems to be related with line https://github.com/wazuh/wazuh/blob/2.1/src/headers/shared.h#L46.

Maybe you could disable that inclusion for AIX:

#if defined(Linux) || defined(FreeBSD) #include <sys/mount.h> #endif

But it's possible that you also have to deal with files https://github.com/wazuh/wazuh/blob/2.1/src/headers/fs_op.h#L38 or https://github.com/wazuh/wazuh/blob/2.1/src/shared/fs_op.c#L39.

They both have the precompiler condition, so I think that you have not to modify them, but I have no AIX installation available right now in order to test it. Please try to make that modification.

Best regards.

rnbwkat commented 6 years ago

The problem is directly related to gcc vs cc. Install gcc and if you need to: ln -s /usr/bin/gcc /usr/bin/cc

When you install gcc, it will bring over the required libraries which fixes the includes. You do not have to comment out any of the includes.

jamesspi commented 6 years ago

Hi @rnbwkat - thank you very much for the feed back, i'll confirm this for you.

@vikman90 - Thanks too. If @rnbwkat's suggestion doesn't work I'll give this a go.

jamesspi commented 6 years ago

@rnbwkat , doesn't this say gcc?

Building shared library libz.so.1.2.8 with /usr/bin/gcc.

rnbwkat commented 6 years ago

My bad - it does. Not sure how you installed gcc. Using rpm? That normally includes all the needed libraries. I can't see what version, but I just ran a vanilla install on a 7.1 system and after using rpm to install gcc and libraries, works like a champ with no mods.

santiago-bassett commented 6 years ago

Here are some notes I took on how to install GCC and compile the agent. This worked on OSSEC 2.8.3, and OSSEC 2.9 and AIX 7.1:

Install GCC:

1.- Download necessary packages from: http://www.perzl.org/aix/index.php?n=Main.Gcc

2.-Install packages: rpm -ivh gcc gcc-cpp libgcc gmp libmpc mpfr libstdc++ If it fails with "AIX-rpm dependency needed", update virtual package: $ /usr/sbin/updtvpkg $ rpm -qa | grep -i aix

3.- Install GMAKE: http://www.perzl.org/aix/index.php?n=Main.Make

4.-Setting up the environment: export CC=/usr/bin/gcc export LD_LIBRARY_PATH=/usr/lib

5.-Compiling agent (OSSEC 2.8.3): $ /usr/local/bin/make setagent $ /usr/local/bin/make all Change user shell in InstallAgent.sh for /bin/bash (/bin/false does not exist) Run install.sh

For OSSEC 2.9.0: Change line 56 for := in Makefile $ /usr/local/bin/make TARGET=agent

Note: libcrypto version is lower than 1.0.1, meaning that agent-auth won't compile for this version. It works when compiling OSSEC 2.6 (then you can copy agent-auth binary)

rnbwkat commented 6 years ago

Couple of things: 3 - try adding make to the rpm install. make is not needed to be made by hand. And the AIX version works. Perhaps not on 2.9 - that is something I need to check.

5 - do not change the shell. Just update security settings in the /etc/shells file to include /bin/false, which does exist. Or in AIX terms, you can change it to /bin/nologin which they use a lot. But making /bin/bash valid for the agent is dangerous.

I will download 2.9 and try again and give a report tomorrow. I have 3 AIX versions/servers running here, so let's have some fun. :-)

jamesspi commented 6 years ago

Hi @vikman90 ,

You were right, need to change

https://github.com/wazuh/wazuh/blob/2.1/src/headers/fs_op.h also:

In file included from ./headers/shared.h:190:0, from client-agent/notify.c:10: ./headers/fs_op.h:41:38: error: dereferencing pointer to incomplete type make: *** [client-agent/notify.o] Error 1

Could you assist in the adjustment that would need to be made please?

Thanks, James

vikman90 commented 6 years ago

Hi @jamesspi,

it seems that FreeBSD or Linux are defined by the environment. Please run this command:

/usr/bin/gcc -dM -E - < /dev/null

It will dump all definitions. Please tell us whether these definitions appear in the output:

FreeBSD Linux AIX

We need this information in order to write the proper condition.

Regards.

jamesspi commented 6 years ago

HI @vikman90,

This is the output:

define DBL_MIN_EXP (-1021)

define UINT_LEAST16_MAX 65535

define __FLT_MIN__ 1.1754943508222875e-38F

define UINT_LEAST8_TYPE unsigned char

define __INTMAX_C(c) c ## LL

define __CHAR_BIT__ 8

define __UINT8_MAX__ 255

define __WINT_MAX__ 2147483647

define ORDER_LITTLE_ENDIAN 1234

define __SIZE_MAX__ 4294967295UL

define __WCHAR_MAX__ 65535

define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1 1

define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2 1

define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 1

define DBL_DENORM_MIN ((double)4.9406564584124654e-324L)

define FLT_EVAL_METHOD 0

define UINT_FAST64_MAX 18446744073709551615ULL

define SIG_ATOMIC_TYPE int

define __DBL_MIN_10_EXP__ (-307)

define FINITE_MATH_ONLY 0

define __GNUC_PATCHLEVEL__ 3

define UINT_FAST8_MAX 255

define DEC64_MAX_EXP 385

define __INT8_C(c) c

define UINT_LEAST64_MAX 18446744073709551615ULL

define __SHRT_MAX__ 32767

define __LDBL_MAX__ 1.7976931348623157e+308L

define UINT_LEAST8_MAX 255

define _ARCH_PPC 1

define __UINTMAX_TYPE__ long long unsigned int

define __DEC32_EPSILON__ 1E-6DF

define __CHAR_UNSIGNED__ 1

define __UINT32_MAX__ 4294967295U

define LDBL_MAX_EXP 1024

define WINT_MIN (-WINT_MAX - 1)

define __SCHAR_MAX__ 127

define __WCHAR_MIN__ 0

define __INT64_C(c) c ## LL

define __DBL_DIG__ 15

define __SIZEOF_INT__ 4

define __SIZEOF_POINTER__ 4

define USER_LABEL_PREFIX

define __STDC_HOSTED__ 1

define LDBL_HAS_INFINITY 1

define _POWER 1

define __FLT_EPSILON__ 1.1920928955078125e-7F

define __LDBL_MIN__ 2.2250738585072014e-308L

define __DEC32_MAX__ 9.999999E96DF

define __INT32_MAX__ 2147483647

define _AIX 1

define __SIZEOF_LONG__ 4

define __UINT16_C(c) c

define __DECIMAL_DIG__ 17

define __LDBL_HAS_QUIET_NAN__ 1

define GNUC 4

define FLT_HAS_DENORM 1

define SIZEOF_LONG_DOUBLE 8

define __BIGGEST_ALIGNMENT__ 16

define __DBL_MAX__ ((double)1.7976931348623157e+308L)

define INT_FAST32_MAX 2147483647

define DBL_HAS_INFINITY 1

define DEC32_MIN_EXP (-94)

define INT_FAST16_TYPE short int

define LDBL_HAS_DENORM 1

define __DEC128_MAX__ 9.999999999999999999999999999999999E6144DL

define INT_LEAST32_MAX 2147483647

define __DEC32_MIN__ 1E-95DF

define DBL_MAX_EXP 1024

define __DEC128_EPSILON__ 1E-33DL

define __PTRDIFF_MAX__ 2147483647L

define _AIX32 1

define _AIX41 1

define _AIX43 1

define _AIX51 1

define _AIX52 1

define _AIX53 1

define LONG_LONG_MAX 9223372036854775807LL

define _AIX61 1

define SIZEOF_SIZE_T 4

define SIZEOF_WINT_T 4

define _IBMR2 1

define __GXX_ABI_VERSION 1002

define FLT_MIN_EXP (-125)

define INT_FAST64_TYPE long long int

define __FP_FAST_FMAF 1

define __FP_FAST_FMAL 1

define __DBL_MIN__ ((double)2.2250738585072014e-308L)

define __DEC128_MIN__ 1E-6143DL

define __REGISTER_PREFIX__

define __UINT16_MAX__ 65535

define DBL_HAS_DENORM 1

define __UINT8_TYPE__ unsigned char

define __NO_INLINE__ 1

define FLT_MANT_DIG 24

define VERSION "4.6.3"

define __UINT64_C(c) c ## ULL

define __BIG_ENDIAN__ 1

define FLOAT_WORD_ORDER ORDER_BIG_ENDIAN

define __INT32_C(c) c

define __DEC64_EPSILON__ 1E-15DD

define ORDER_PDP_ENDIAN 3412

define DEC128_MIN_EXP (-6142)

define INT_FAST32_TYPE int

define UINT_LEAST16_TYPE short unsigned int

define __INT16_MAX__ 32767

define __SIZE_TYPE__ long unsigned int

define __UINT64_MAX__ 18446744073709551615ULL

define __INT8_TYPE__ signed char

define __HAVE_BSWAP__ 1

define __FLT_RADIX__ 2

define INT_LEAST16_TYPE short int

define __LDBL_EPSILON__ 2.2204460492503131e-16L

define __UINTMAX_C(c) c ## ULL

define _LONG_LONG 1

define SIG_ATOMIC_MAX 2147483647

define SIZEOF_PTRDIFF_T 4

define DEC32_SUBNORMAL_MIN 0.000001E-95DF

define INT_FAST16_MAX 32767

define UINT_FAST32_MAX 4294967295U

define UINT_LEAST64_TYPE long long unsigned int

define __FLT_HAS_QUIET_NAN__ 1

define __FLT_MAX_10_EXP__ 38

define __LONG_MAX__ 2147483647L

define DEC128_SUBNORMAL_MIN 0.000000000000000000000000000000001E-6143DL

define FLT_HAS_INFINITY 1

define UINT_FAST16_TYPE short unsigned int

define __DEC64_MAX__ 9.999999999999999E384DD

define __CHAR16_TYPE__ short unsigned int

define __PRAGMA_REDEFINE_EXTNAME 1

define INT_LEAST16_MAX 32767

define DEC64_MANT_DIG 16

define __INT64_MAX__ 9223372036854775807LL

define UINT_LEAST32_MAX 4294967295U

define INT_LEAST64_TYPE long long int

define __INT16_TYPE__ short int

define INT_LEAST8_TYPE signed char

define DEC32_MAX_EXP 97

define _BIG_ENDIAN 1

define INT_FAST8_MAX 127

define __INTPTR_MAX__ 2147483647L

define LDBL_MANT_DIG 53

define __DBL_HAS_QUIET_NAN__ 1

define SIG_ATOMIC_MIN (-SIG_ATOMIC_MAX - 1)

define __INTPTR_TYPE__ long int

define __UINT16_TYPE__ short unsigned int

define __WCHAR_TYPE__ short unsigned int

define __SIZEOF_FLOAT__ 4

define __UINTPTR_MAX__ 4294967295UL

define DEC64_MIN_EXP (-382)

define INT_FAST64_MAX 9223372036854775807LL

define __FLT_DIG__ 6

define UINT_FAST64_TYPE long long unsigned int

define __INT_MAX__ 2147483647

define __INT64_TYPE__ long long int

define FLT_MAX_EXP 128

define DBL_MANT_DIG 53

define INT_LEAST64_MAX 9223372036854775807LL

define __DEC64_MIN__ 1E-383DD

define __WINT_TYPE__ int

define UINT_LEAST32_TYPE unsigned int

define __SIZEOF_SHORT__ 2

define LDBL_MIN_EXP (-1021)

define INT_LEAST8_MAX 127

define __LDBL_MAX_10_EXP__ 308

define __DBL_EPSILON__ ((double)2.2204460492503131e-16L)

define __UINT8_C(c) c

define INT_LEAST32_TYPE int

define SIZEOF_WCHAR_T 2

define __UINT64_TYPE__ long long unsigned int

define INT_FAST8_TYPE signed char

define DBL_DECIMAL_DIG 17

define DEC_EVAL_METHOD 2

define ORDER_BIG_ENDIAN 4321

define __UINT32_C(c) c ## U

define __INTMAX_MAX__ 9223372036854775807LL

define __BYTE_ORDER ORDER_BIG_ENDIAN__

define FLT_DENORM_MIN 1.4012984643248171e-45F

define __INT8_MAX__ 127

define UINT_FAST32_TYPE unsigned int

define __CHAR32_TYPE__ unsigned int

define __FLT_MAX__ 3.4028234663852886e+38F

define __FP_FAST_FMA 1

define __INT32_TYPE__ int

define __SIZEOF_DOUBLE__ 8

define __FLT_MIN_10_EXP__ (-37)

define __INTMAX_TYPE__ long long int

define DEC128_MAX_EXP 6145

define __GNUC_MINOR__ 6

define __UINTMAX_MAX__ 18446744073709551615ULL

define DEC32_MANT_DIG 7

define __DBL_MAX_10_EXP__ 308

define LDBL_DENORM_MIN 4.9406564584124654e-324L

define __INT16_C(c) c

define STDC 1

define __PTRDIFF_TYPE__ long int

define __UINT32_TYPE__ unsigned int

define __UINTPTR_TYPE__ long unsigned int

define DEC64_SUBNORMAL_MIN 0.000000000000001E-383DD

define DEC128_MANT_DIG 34

define __LDBL_MIN_10_EXP__ (-307)

define SIZEOF_LONG_LONG 8

define __LDBL_DIG__ 15

define FLT_DECIMAL_DIG 9

define UINT_FAST16_MAX 65535

define GNUC_GNU_INLINE 1

define UINT_FAST8_TYPE unsigned char

Thanks, James

vikman90 commented 6 years ago

Hi,

I cannot find neither FreeBSD nor Linux. This is OK.

Which branch are you using?

If you are using branch master or stable, this problem should be fixed by changing line https://github.com/wazuh/wazuh-dev/blob/master/src/headers/fs_op.h#L39:

#else

for:

#elif defined(Linux) || defined(FreeBSD)

If you are using branch 2.0 or 2.1 (that I don't recommend you to use because they are development branches) they should have this patch already. If the problem persists, please try to change line https://github.com/wazuh/wazuh-dev/blob/2.0/src/headers/fs_op.h#L38:

#elif !defined(WIN32) && (defined(Linux) || defined(FreeBSD))

for:

#elif !defined(WIN32) && (defined(Linux) || defined(FreeBSD)) && !defined(_AIX)

I hope this help you.

Best, Victor.

jamesspi commented 6 years ago

Thanks @vikman90 will give this a go.

jamesspi commented 6 years ago

Hi @vikman90 ,

I now get:

Para instalação em português, escolha [br]. 要使用中文进行安装, 请选择 [cn]. Für eine deutsche Installation, wählen Sie [de]. Για εγκατάσταση στα Ελληνικά, επιλέξτε [el]. For installation in English, choose [en]. Para instalar en español, elija [es]. Pour une installation en français, choisissez [fr] A Magyar nyelvű telepítéshez válassza [hu]. Per l'installazione in Italiano, scegli [it]. 日本語でインストールします.選択して下さい.[jp]. Voor installatie in het Nederlands, kies [nl]. Aby instalować w języku Polskim, wybierz [pl]. Для инструкций по установке на русском ,введите [ru]. Za instalaciju na srpskom, izaberi [sr]. ** Türkçe kurulum için seçin [tr]. (en/br/cn/de/el/es/fr/hu/it/jp/nl/pl/ru/sr/tr) [en]:

 Wazuh v2.0.1 Installation Script - http://www.wazuh.com

You are about to start the installation process of Wazuh. You must have a C compiler pre-installed in your system.

1- What kind of installation do you want (server, agent, local, hybrid or help)?

2- Setting up the installation environment.

3- Configuring Wazuh.

3.1- What's the IP Address or hostname of the Wazuh server?:

4- Installing the system

jamesspi commented 6 years ago

Hi @vikman90,

Have you had a chance to look at my previous comment by any chance?

Thanks.

jamesspi commented 6 years ago

Anyone?

vikman90 commented 6 years ago

Hi @jamesspi,

I got an AIX 6.1 installation and will try to reproduce the issue ASAP.

Regards, Victor.

jamesspi commented 6 years ago

Thank you @vikman90, appreciate it!

vikman90 commented 6 years ago

@jamesspi we did it!

I had to make some changes in the code:

  1. Restrict mounth.h to be included only for Linux and FreeBSD: https://github.com/wazuh/wazuh/commit/b014254e26f488fb5243389be169735244c9a098.

  2. Prevent struct statfs from being used on other systems than Linux or FreeBSD: https://github.com/wazuh/wazuh/commit/eaec6be279c2ff2e6959deb6bac7e45e02abf452#diff-913ffcf195b8fe8cb27eb88ea08bdbe9R40. You already fixed it, and it's fixed in branch 2.0

  3. I got an error close to the one that you have, and it seems to be caused by the definition of __version. It should be refactored: https://github.com/wazuh/wazuh/commit/8faf377980b81945bef12624be00076939e07ca4

  4. The code was compiled with no pthread option, and it should be added: https://github.com/wazuh/wazuh/commit/a3b77431c501006c789105e0a6a2b9af22dd8d15

  5. AIX defines the type name label_t, and Authd re-defines it. The type name (but not the structure) is superfluous and may be removed: https://github.com/wazuh/wazuh/commit/1954d083436dfa3745ef0f0a36f63bffd6185548

  6. (Only to compile the manager) Monitord uses getopt.h to parse a long option, --no-agents. It may be replaced for -n and this way we may remove the troubling inclusion: https://github.com/wazuh/wazuh/commit/8e5897151ac1b45c92ed61caad65631edbd02d11

All this commits have been made in branch 2.0. I've compiled it on AIX 6.1 with GCC 4.9.3.

I hope this solve your problem.

Best regards, Victor.

jamesspi commented 6 years ago

@vikman90 fantastic, thank you so much! I will give this a go.

jamesspi commented 6 years ago

Hi @vikman90,

Just to let you know that compilation worked :)

I had to make two changes to the src/init/adduser.sh script though:

Replace /usr/sbin/mkgroup with /usr/bin/mkgroup Reflect the useradd options to be the same as SunOS.

I have asked my AIX engineer to try again, hopefully this is all that's left.

Thanks again, James

jamesspi commented 6 years ago

@vikman90 - I forgot to mention that I also got this make: *** [install-common] Error 127

All error lines looked like:

./init/adduser.sh ossec ossecm ossecr ossec /var/ossec Wait for success... ./init/adduser.sh[61]: /usr/sbin/mkgroup: not found make: *** [install-common] Error 127 chown: ossec is an unknown groupname. ln: No such file or directory

I adjusted the mkgroup path and adduser options - just to make sure, is there something else that would cause make to fail?

Thanks, James

vikman90 commented 6 years ago

Hi @jamesspi,

you are right, we had a problem with useradd on Solaris some days before. Thanks for notice us, if you have any other problem, let me know and I will try to get an AIX system again to test it.

Regards.

jamesspi commented 6 years ago

Hi @vikman90 ,

I can confirm the useradd and mkgroup changes work, but now - it seems that the AIX version of /usr/bin/install accepts different arguments:

install -d -m 0750 -o root -g ossec /var/ossec/ /usr/bin/getopt: illegal option -- d Usage: install [-c dira] [-f dirb] [-i] [-m] [-M mode] [-O owner] [-G group] [-S] [-n dirc] [-o] [-s] file [dirx ...] make: *** [install-common] Error 2

I guess I would have to change all instances of install in the Makefile, correct? I don't see a clause in there for an AIX uname.

Thanks, James

vikman90 commented 6 years ago

Hi,

please take a look to this commit: https://github.com/wazuh/wazuh/commit/bf7d3f486a6a3f31314ffac000f7e4b07221facf

We found a close (maybe the same) problem on Solaris and changed install for ginstall. Maybe AIX has also ginstall and could be solved in the same way.

Regards.

jamesspi commented 6 years ago

Hi @vikman90,

Don't think so - there is installbsd, but parameters are different still.

Thanks, James

aremai commented 6 years ago

Hi all,

@vikman90 ginstall is part of the coreutils package on Solaris and is needed to compile wazuh on Solaris properly. The native gcc and make and install are not the same gnu-tools that are used on Linux systems. They behave absurdly differently and only work with Solaris-native packages.... The coreutils package is a 3rd party package and provides all different gnu-tools... https://www.opencsw.org/search/coreutils/

cheers, theresa

vikman90 commented 6 years ago

Hi @jamesspi and @aremai,

thank you for your clarification! It fails for create directories and I guess that it will also fail for files (-m becomes -M and -o becomes -O). Long time ago the installation procedure was made by a script (instead of Makefile).

I think that taking installation back to a script would make easier to fix this problem. We had this idea in mind for another project. I will discuss about it with the development team and we will propose a solution.

In the meantime, ideas are welcome!

Best regards, Victor.