threathunters-io / laurel

Transform Linux Audit logs for SIEM usage
GNU General Public License v3.0
710 stars 56 forks source link

Issues installing on C7 #86

Closed 100110010111 closed 2 years ago

100110010111 commented 2 years ago

Any ideas?

cargo build --release Compiling laurel v0.4.0 (/opt/laurel) error: failed to run custom build command for laurel v0.4.0 (/opt/laurel)

Caused by: process didn't exit successfully: /opt/laurel/target/release/build/laurel-8d418fa9b5624d53/build-script-build (exit status: 101) --- stderr /usr/include/netinet/in.h:32:8: error: redefinition of 'in_addr' /usr/include/linux/in.h:84:8: note: previous definition is here /usr/include/bits/in.h:155:8: error: redefinition of 'ip_mreqn' /usr/include/linux/in.h:171:8: note: previous definition is here /usr/include/bits/in.h:163:8: error: redefinition of 'in_pktinfo' /usr/include/linux/in.h:220:8: note: previous definition is here /usr/include/netinet/in.h:43:5: error: redefinition of enumerator 'IPPROTO_IP' /usr/include/linux/in.h:28:3: note: previous definition is here /usr/include/netinet/in.h:45:5: error: redefinition of enumerator 'IPPROTO_ICMP' /usr/include/linux/in.h:30:3: note: previous definition is here /usr/include/netinet/in.h:47:5: error: redefinition of enumerator 'IPPROTO_IGMP' /usr/include/linux/in.h:32:3: note: previous definition is here /usr/include/netinet/in.h:49:5: error: redefinition of enumerator 'IPPROTO_IPIP' /usr/include/linux/in.h:34:3: note: previous definition is here /usr/include/netinet/in.h:51:5: error: redefinition of enumerator 'IPPROTO_TCP' /usr/include/linux/in.h:36:3: note: previous definition is here /usr/include/netinet/in.h:53:5: error: redefinition of enumerator 'IPPROTO_EGP' /usr/include/linux/in.h:38:3: note: previous definition is here /usr/include/netinet/in.h:55:5: error: redefinition of enumerator 'IPPROTO_PUP' /usr/include/linux/in.h:40:3: note: previous definition is here /usr/include/netinet/in.h:57:5: error: redefinition of enumerator 'IPPROTO_UDP' /usr/include/linux/in.h:42:3: note: previous definition is here /usr/include/netinet/in.h:59:5: error: redefinition of enumerator 'IPPROTO_IDP' /usr/include/linux/in.h:44:3: note: previous definition is here /usr/include/netinet/in.h:61:5: error: redefinition of enumerator 'IPPROTO_TP' /usr/include/linux/in.h:46:3: note: previous definition is here /usr/include/netinet/in.h:63:5: error: redefinition of enumerator 'IPPROTO_DCCP' /usr/include/linux/in.h:48:3: note: previous definition is here /usr/include/netinet/in.h:65:5: error: redefinition of enumerator 'IPPROTO_IPV6' /usr/include/linux/in.h:50:3: note: previous definition is here /usr/include/netinet/in.h:67:5: error: redefinition of enumerator 'IPPROTO_RSVP' /usr/include/linux/in.h:52:3: note: previous definition is here /usr/include/netinet/in.h:69:5: error: redefinition of enumerator 'IPPROTO_GRE' /usr/include/linux/in.h:54:3: note: previous definition is here /usr/include/netinet/in.h:71:5: error: redefinition of enumerator 'IPPROTO_ESP' /usr/include/linux/in.h:56:3: note: previous definition is here /usr/include/netinet/in.h:73:5: error: redefinition of enumerator 'IPPROTO_AH' /usr/include/linux/in.h:58:3: note: previous definition is here fatal error: too many errors emitted, stopping now [-ferror-limit=] /usr/include/netinet/in.h:32:8: error: redefinition of 'in_addr', err: true /usr/include/bits/in.h:155:8: error: redefinition of 'ip_mreqn', err: true /usr/include/bits/in.h:163:8: error: redefinition of 'in_pktinfo', err: true /usr/include/netinet/in.h:43:5: error: redefinition of enumerator 'IPPROTO_IP', err: true /usr/include/netinet/in.h:45:5: error: redefinition of enumerator 'IPPROTO_ICMP', err: true /usr/include/netinet/in.h:47:5: error: redefinition of enumerator 'IPPROTO_IGMP', err: true /usr/include/netinet/in.h:49:5: error: redefinition of enumerator 'IPPROTO_IPIP', err: true /usr/include/netinet/in.h:51:5: error: redefinition of enumerator 'IPPROTO_TCP', err: true /usr/include/netinet/in.h:53:5: error: redefinition of enumerator 'IPPROTO_EGP', err: true /usr/include/netinet/in.h:55:5: error: redefinition of enumerator 'IPPROTO_PUP', err: true /usr/include/netinet/in.h:57:5: error: redefinition of enumerator 'IPPROTO_UDP', err: true /usr/include/netinet/in.h:59:5: error: redefinition of enumerator 'IPPROTO_IDP', err: true /usr/include/netinet/in.h:61:5: error: redefinition of enumerator 'IPPROTO_TP', err: true /usr/include/netinet/in.h:63:5: error: redefinition of enumerator 'IPPROTO_DCCP', err: true /usr/include/netinet/in.h:65:5: error: redefinition of enumerator 'IPPROTO_IPV6', err: true /usr/include/netinet/in.h:67:5: error: redefinition of enumerator 'IPPROTO_RSVP', err: true /usr/include/netinet/in.h:69:5: error: redefinition of enumerator 'IPPROTO_GRE', err: true /usr/include/netinet/in.h:71:5: error: redefinition of enumerator 'IPPROTO_ESP', err: true /usr/include/netinet/in.h:73:5: error: redefinition of enumerator 'IPPROTO_AH', err: true fatal error: too many errors emitted, stopping now [-ferror-limit=], err: true thread 'main' panicked at 'unable to generate bindings: ()', build.rs:114:10 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

hillu commented 2 years ago

Can you build laurel if you comment out or remove the line

#include <sys/socket.h>

at the top of src/sockaddr.h ?

hillu commented 2 years ago

I gave it a shot on a CentOS7 box.

Removing the sys/socket.h include did not help. Removing linux/in.h does help. However, some other (currently unneeded) header files that don't exist on CentOS7 have to be removed, namely linux/llc.h, linux/qrtr.h, linux/rxrpc.h.

At this point I ran into problems with an old Clang version (3.9 or higher being required). I was able to fix this by downgrading the bindgen dependency to 0.54, renaming "allowlist" to "whitelist", and removing the rust_target call in build.rs.

I don't think I want to commit this to master right now, but I'd be glad if it helps in your case.

100110010111 commented 2 years ago

Is this where you changed from allowlist to whitelist (from build.rs)?

    .allowlist_type("^sockaddr_.*")
    .allowlist_var("^AF_.*")
hillu commented 2 years ago

Is this where you changed from allowlist to whitelist (from build.rs)?

    .allowlist_type("^sockaddr_.*")
    .allowlist_var("^AF_.*")

yes. Here's the entire diff:

diff --git a/Cargo.toml b/Cargo.toml
index 7d2b88f..3d25318 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -30,7 +30,7 @@ posix-acl = ">= 1"
 regex = ">= 1.5.0"

 [build-dependencies]
-bindgen = ">= 0.56"
+bindgen = "~ 0.54"
 # cc = ">= 1"

 [dev-dependencies]
diff --git a/build.rs b/build.rs
index 2f19361..78d53a9 100644
--- a/build.rs
+++ b/build.rs
@@ -105,9 +105,8 @@ fn main() -> Result<(),Box<dyn std::error::Error>> {
     // sockaddr
     bindgen::Builder::default()
         .header("src/sockaddr.h")
-        .rust_target(bindgen::RustTarget::Stable_1_47)
-        .allowlist_type("^sockaddr_.*")
-        .allowlist_var("^AF_.*")
+        .whitelist_type("^sockaddr_.*")
+        .whitelist_var("^AF_.*")
         .layout_tests(false)
         .rustfmt_bindings(false)
         .generate()
diff --git a/src/sockaddr.h b/src/sockaddr.h
index 4102d5b..15d3211 100644
--- a/src/sockaddr.h
+++ b/src/sockaddr.h
@@ -1,3 +1,6 @@
+
+// #define _NETINET_IN_H   1
+
 #include <sys/socket.h>

 #include <linux/atalk.h>
@@ -11,21 +14,21 @@
 // #include <linux/if_pppox.h>
 // #include <linux/if_xdp.h>
 #include <linux/in6.h>
-#include <linux/in.h>
+// #include <linux/in.h>
 #include <linux/l2tp.h>
-#include <linux/llc.h>
+// #include <linux/llc.h>
 #include <linux/netlink.h>
 #include <linux/nfc.h>
 #include <linux/phonet.h>
-#include <linux/qrtr.h>
+// #include <linux/qrtr.h>
 #include <linux/rose.h>
-#include <linux/rxrpc.h>
+// #include <linux/rxrpc.h>
 #include <linux/tipc.h>
 #include <linux/un.h>
 #include <linux/vm_sockets.h>
 #include <linux/x25.h>

-#include <inttypes.h>
+// #include <inttypes.h>

 /* Apparently, ipx.h is no longer generally available. */
100110010111 commented 2 years ago

Thanks- I have that and am getting further but still getting "This crrate requires that libclang 3.9 or later be installed on this system. Maybe ill bite the bullet and try to do that.

hillu commented 2 years ago

Thanks- I have that and am getting further but still getting "This crrate requires that libclang 3.9 or later be installed on this system. Maybe ill bite the bullet and try to do that.

Strange, this worked in my environment. Have you got llvm and clang (version 3.4.2-9.el7 from the regular centos7 repo) installed?

100110010111 commented 2 years ago

I do.

hillu commented 2 years ago

perhaps running cargo clean and rebuilding everything from scratch will help?

100110010111 commented 2 years ago

Same error:

Compiling toml v0.5.8 warning: laurel (build script) generated 2 warnings error: failed to run custom build command for laurel v0.4.0 (/opt/laurel)

Caused by: process didn't exit successfully: /opt/laurel/target/release/build/laurel-8d418fa9b5624d53/build-script-build (exit status: 101) --- stderr thread 'main' panicked at 'libclang function not loaded: clang_Type_getNumTemplateArguments. This crate requires that libclang 3.9 or later be installed on your system. For more information on how to accomplish this, see here: https://rust-lang.github.io/rust-bindgen/requirements.html#installing-clang-39', /home/user/gitroot/rust/cargo/registry/src/rust-d47c9d027e7114f9/clang-sys-1.3.0/src/lib.rs:1704:1 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

100110010111 commented 2 years ago

I was able to use the precompiled 0.41 binary. Thanks

sraue commented 2 years ago

Is this where you changed from allowlist to whitelist (from build.rs)?

    .allowlist_type("^sockaddr_.*")
    .allowlist_var("^AF_.*")

yes. Here's the entire diff:

diff --git a/Cargo.toml b/Cargo.toml
index 7d2b88f..3d25318 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -30,7 +30,7 @@ posix-acl = ">= 1"
 regex = ">= 1.5.0"

 [build-dependencies]
-bindgen = ">= 0.56"
+bindgen = "~ 0.54"
 # cc = ">= 1"

 [dev-dependencies]
diff --git a/build.rs b/build.rs
index 2f19361..78d53a9 100644
--- a/build.rs
+++ b/build.rs
@@ -105,9 +105,8 @@ fn main() -> Result<(),Box<dyn std::error::Error>> {
     // sockaddr
     bindgen::Builder::default()
         .header("src/sockaddr.h")
-        .rust_target(bindgen::RustTarget::Stable_1_47)
-        .allowlist_type("^sockaddr_.*")
-        .allowlist_var("^AF_.*")
+        .whitelist_type("^sockaddr_.*")
+        .whitelist_var("^AF_.*")
         .layout_tests(false)
         .rustfmt_bindings(false)
         .generate()
diff --git a/src/sockaddr.h b/src/sockaddr.h
index 4102d5b..15d3211 100644
--- a/src/sockaddr.h
+++ b/src/sockaddr.h
@@ -1,3 +1,6 @@
+
+// #define _NETINET_IN_H   1
+
 #include <sys/socket.h>

 #include <linux/atalk.h>
@@ -11,21 +14,21 @@
 // #include <linux/if_pppox.h>
 // #include <linux/if_xdp.h>
 #include <linux/in6.h>
-#include <linux/in.h>
+// #include <linux/in.h>
 #include <linux/l2tp.h>
-#include <linux/llc.h>
+// #include <linux/llc.h>
 #include <linux/netlink.h>
 #include <linux/nfc.h>
 #include <linux/phonet.h>
-#include <linux/qrtr.h>
+// #include <linux/qrtr.h>
 #include <linux/rose.h>
-#include <linux/rxrpc.h>
+// #include <linux/rxrpc.h>
 #include <linux/tipc.h>
 #include <linux/un.h>
 #include <linux/vm_sockets.h>
 #include <linux/x25.h>

-#include <inttypes.h>
+// #include <inttypes.h>

 /* Apparently, ipx.h is no longer generally available. */

Hi @hillu,

can you include this patch? with this patch i was able to build on Centos 7.

can you build the glibc shared builds based on centos 7 or similiar in the future, they seem to work on newer SLES 15 systems too, and we have to use laurel on systems with at least centos 7 /RHEL 7 and newer

hillu commented 2 years ago

Oh well. Building on Debian 9 works using a compiler installed from rustup if bindgen is pinned to a pre-0.60 version. Unfortunately, the resulting x86_64 binary requires glibc 2.18, this is not low enough for RHEL 7.

hillu commented 2 years ago

@sraue Got it, we'll build on CentOS7 (but using non-ancient clang and modern rustc), starting with the next release. If you need binaries for 0.4.1 now, please have a look at what is done in release.yml.

sraue commented 2 years ago

@hillu, many thanks for your fast help. i will test and compile byself until the next release.