search

owasp-modsecurity / ModSecurity

ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.
https://www.modsecurity.org
Apache License 2.0
8.28k stars 1.61k forks source link

Compile Modsecurity v3.x for WASM architecture #2590

Open M4tteoP opened 3 years ago

M4tteoP commented 3 years ago

Using emscripten (emconfigure and emmake) I am working on compiling the library libmodsecurity for WebAssembly. Referring to this issue and this solution, the main problem that I'm facing is that I am including paths to system headers (-I/usr/include -I/usr/include/libxml2). Could anyone help me to figure out how to modify the MakeFile to avoid these inclusions? The command that I should change is: 

/home/user/Tesi/extensions/manualBuild/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=25 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -fPIC -DPIC parser/seclang-parser.cc -o parser/.libs/libmodsecurity_la-seclang-parser.o

Action performed are just: ./build.sh emconfigure ./configure emmake make Resulting in this error:

/bin/bash ../libtool  --tag=CXX   --mode=compile /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/em++ -DHAVE_CONFIG_H -I.  -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include          -I/usr/include/libxml2 -DWITH_LIBXML2   -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -o parser/libmodsecurity_la-seclang-parser.lo `test -f 'parser/seclang-parser.cc' || echo './'`parser/seclang-parser.cc
libtool: compile:  /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/em++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c parser/seclang-parser.cc  -fPIC -DPIC -o parser/.libs/libmodsecurity_la-seclang-parser.o
In file included from parser/seclang-parser.cc:41:
In file included from seclang-parser.yy:11:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/string:511:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/string_view:179:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__string:57:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/algorithm:651:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/cstring:60:
/usr/include/string.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
#include <bits/libc-header-start.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
em++: error: '/home/user/Tesi/extensions/manualBuild/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=25 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -fPIC -DPIC parser/seclang-parser.cc -o parser/.libs/libmodsecurity_la-seclang-parser.o' failed (returned 1)
make[3]: *** [Makefile:2072: parser/libmodsecurity_la-seclang-parser.lo] Error 1
make[3]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make[2]: *** [Makefile:3479: all-recursive] Error 1
make[2]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make[1]: *** [Makefile:1224: all] Error 2
make[1]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make: *** [Makefile:1035: all-recursive] Error 1
emmake: error: 'make' failed (returned 2)

Thanks for any help about the generation and the possibility to change this building phase

zimmerle commented 3 years ago

Hi @M4tteoP, that sounds like an exciting project. What distro do you have? Is that the latest emsdk? I am trying to replicate the environment.

M4tteoP commented 3 years ago

hi @zimmerle, thanks a lot for your interest. I am working on Ubuntu 20.04.2, emsdk is the latest one (2.0.25), just cloned a few days ago from GitHub.

I post the result of emconfigure ./configure to show you all the dependencies installed:

ModSecurity - v3.0.4-131-gfaad65d3 for Linux

 Mandatory dependencies
   + libInjection                                  ....v3.9.2-46-gbfba51f
   + SecLang tests                                 ....a3d4405

 Optional dependencies
   + GeoIP/MaxMind                                 ....found
      * (GeoIP) v
         /usr/lib/x86_64-linux-gnu//libGeoIP.so, /usr/include
   + LibCURL                                       ....found v7.68.0
      -lcurl,  -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL
   + YAJL                                          ....not found
   + LMDB                                          ....not found
   + LibXML2                                       ....found v2.9.10
      -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2
   + SSDEEP                                        ....not found
   + LUA                                           ....not found

 Other Options
   + Test Utilities                                ....disabled
   + SecDebugLog                                   ....enabled
   + afl fuzzer                                    ....disabled
   + library examples                              ....enabled
   + Building parser                               ....disabled
   + Treating pm operations as critical section    ....disabled

To contextualize a bit the project, I'm working on implementing a security control based on Modsecurity as an Istio extension. It requires to have a wasm file with the whole logic, so the idea was to:

  1. Build the ModSecurity library for WASM (hopefully statically?).
  2. Link it to a small c++ project based on the SDK to interact with the proxy.

Thanks, I'll be available to provide you any further detail about the environment.

M4tteoP commented 3 years ago

I let you know that I have updated ModSecurity, I am now working with v3.0.5-2-g465db29b. I tried to manually edit the MakeFile removing:

at line 488: GEOIP_CFLAGS = -DWITH_GEOIP -I/usr/include

at line 512: LIBXML2_CFLAGS = -I/usr/include/libxml2 -DWITH_LIBXML2

at line 515: LIBXML2_DISPLAY = -lxml2, -I/usr/include/libxml2 -DWITH_LIBXML2

But still the command executed and relative inclusion errors are the same.

/bin/bash ../libtool  --tag=CXX   --mode=compile /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/em++ -DHAVE_CONFIG_H -I.  -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include          -I/usr/include/libxml2 -DWITH_LIBXML2   -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -o parser/libmodsecurity_la-seclang-parser.lo `test -f 'parser/seclang-parser.cc' || echo './'`parser/seclang-parser.cc
libtool: compile:  /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/em++ -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c parser/seclang-parser.cc  -fPIC -DPIC -o parser/.libs/libmodsecurity_la-seclang-parser.o
In file included from parser/seclang-parser.cc:41:
In file included from seclang-parser.yy:11:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/string:511:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/string_view:179:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__string:57:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/algorithm:651:
In file included from /home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/cstring:60:
/usr/include/string.h:26:10: fatal error: 'bits/libc-header-start.h' file not found
#include <bits/libc-header-start.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
em++: error: '/home/user/Tesi/extensions/manualBuild/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=25 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/home/user/Tesi/extensions/manualBuild/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -DHAVE_CONFIG_H -I. -std=c++11 -I.. -g -I../others -fPIC -O3 -I../headers -DWITH_CURL_SSLVERSION_TLSv1_2 -DWITH_CURL -DWITH_GEOIP -I/usr/include -I/usr/include/libxml2 -DWITH_LIBXML2 -g -O2 -MT parser/libmodsecurity_la-seclang-parser.lo -MD -MP -MF parser/.deps/libmodsecurity_la-seclang-parser.Tpo -c -fPIC -DPIC parser/seclang-parser.cc -o parser/.libs/libmodsecurity_la-seclang-parser.o' failed (returned 1)
make[3]: *** [Makefile:2072: parser/libmodsecurity_la-seclang-parser.lo] Error 1
make[3]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make[2]: *** [Makefile:3479: all-recursive] Error 1
make[2]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make[1]: *** [Makefile:1224: all] Error 2
make[1]: Leaving directory '/home/user/Tesi/extensions/manualBuild/ModSecurity/src'
make: *** [Makefile:1035: all-recursive] Error 1
emmake: error: 'make' failed (returned 2)
daixiang0 commented 2 years ago

@martinhsv Hi, where would be the best place to post if I have completed implementation for WASM? A new dir named wasm?

M4tteoP commented 2 years ago

Hi @daixiang0 , it there actually a fork where your work is already published? Hopefully it will become a thing here in the official repo sooner or later

daixiang0 commented 2 years ago

I will post a PR soon and hope it can be merged.

daixiang0 commented 2 years ago

We have created a project https://github.com/intel/modsecurity-wasm-filter which introduce ModSecurity into Service Mesh.

M4tteoP commented 2 years ago

Thanks @daixiang0! I already took a look at the intel repo, I was not sure that you were part of the same team! Speaking about the Modsecurity WASM support, do you have any plans about including some other libraries and eventually the excluded features (but still, I'm aware that first of all limitations come from the WASM itself, maybe the workarounds that you did are the only way to go)?

daixiang0 commented 2 years ago

We are from the same team, you can check the PR and do some try, not sure it is the only way.