ADD-SP / ngx_waf

Handy, High performance, ModSecurity compatible Nginx firewall module & 方便、高性能、兼容 ModSecurity 的 Nginx 防火墙模块
https://add-sp.github.io/ngx_waf-docs/
BSD 3-Clause "New" or "Revised" License
1.49k stars 188 forks source link

alpine build error #17

Closed phpstatic closed 3 years ago

phpstatic commented 3 years ago
In file included from ../../module/ngx_waf/src/ngx_http_waf_module_core.c:1:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_core.h:17:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_check.h:15:
../../module/ngx_waf/inc/ngx_http_waf_module_util.h:191:27: error: no member named '__in6_u' in 'struct in6_addr'
    memcpy(prefix, &addr6.__in6_u.__u6_addr8, 16);
                    ~~~~~ ^
In file included from ../../module/ngx_waf/src/ngx_http_waf_module_core.c:1:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_core.h:17:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_check.h:17:
../../module/ngx_waf/inc/ngx_http_waf_module_ip_trie.h:173:42: error: no member named '__in6_u' in 'struct in6_addr'
            if (CHECK_BIT(inx_addr->ipv6.__in6_u.__u6_addr8[uint8_index], 7 - (bit_index % 8)) != TRUE) {
                          ~~~~~~~~~~~~~~ ^
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:295:51: note: expanded from macro 'CHECK_BIT'
#define CHECK_BIT(origin, bit_index) (CHECK_FLAG((origin), 1 << (bit_index)))
                                                  ^~~~~~
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:284:37: note: expanded from macro 'CHECK_FLAG'
#define CHECK_FLAG(origin, flag) (((origin) & (flag)) != 0 ? TRUE : FALSE)
                                    ^~~~~~
In file included from ../../module/ngx_waf/src/ngx_http_waf_module_core.c:1:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_core.h:17:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_check.h:17:
../../module/ngx_waf/inc/ngx_http_waf_module_ip_trie.h:194:38: error: no member named '__in6_u' in 'struct in6_addr'
        if (CHECK_BIT(inx_addr->ipv6.__in6_u.__u6_addr8[uint8_index], 7 - (bit_index % 8)) != TRUE) {
                      ~~~~~~~~~~~~~~ ^
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:295:51: note: expanded from macro 'CHECK_BIT'
#define CHECK_BIT(origin, bit_index) (CHECK_FLAG((origin), 1 << (bit_index)))
                                                  ^~~~~~
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:284:37: note: expanded from macro 'CHECK_FLAG'
#define CHECK_FLAG(origin, flag) (((origin) & (flag)) != 0 ? TRUE : FALSE)
                                    ^~~~~~
In file included from ../../module/ngx_waf/src/ngx_http_waf_module_core.c:1:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_core.h:17:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_check.h:17:
../../module/ngx_waf/inc/ngx_http_waf_module_ip_trie.h:235:42: error: no member named '__in6_u' in 'struct in6_addr'
            if (CHECK_BIT(inx_addr->ipv6.__in6_u.__u6_addr8[uint8_index], 7 - (bit_index % 8)) != TRUE) {
                          ~~~~~~~~~~~~~~ ^
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:295:51: note: expanded from macro 'CHECK_BIT'
#define CHECK_BIT(origin, bit_index) (CHECK_FLAG((origin), 1 << (bit_index)))
                                                  ^~~~~~
../../module/ngx_waf/inc/ngx_http_waf_module_macro.h:284:37: note: expanded from macro 'CHECK_FLAG'
#define CHECK_FLAG(origin, flag) (((origin) & (flag)) != 0 ? TRUE : FALSE)
                                    ^~~~~~
In file included from ../../module/ngx_waf/src/ngx_http_waf_module_core.c:1:
In file included from ../../module/ngx_waf/inc/ngx_http_waf_module_core.h:18:
../../module/ngx_waf/inc/ngx_http_waf_module_config.h:522:34: error: no member named '__in6_u' in 'struct in6_addr'
            memcpy(inx_addr.ipv6.__in6_u.__u6_addr8, ipv6.prefix, 16);
                   ~~~~~~~~~~~~~ ^
ADD-SP commented 3 years ago

Thanks for pointing out this bug caused by some undefined behavior in the code.

I have uploaded the hotfix commit and you can execute the following command to recompile the installation. If there are no errors please close the issue.

git checkout master
git pull --rebase
git checkout -b hotfix-v3.0.1 origin/hotfix-v3.0.1
phpstatic commented 3 years ago

build error fixed.

I am not sure how to setup the the cc and ipv4 black list to work. here is my config file in server section:

listen 88;
waf on;
waf_mode STD;
waf_rule_path /etc/nginx/waf/rules/;
waf_cc_deny_limit 10 60;

test ok and restart:

nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
/etc/init.d/nginx restart
[ ok ] Restarting nginx (via systemctl): nginx.service.

and also put my ip into /etc/nginx/waf/rules/ipv4, return 200.

test with wrk -t1 -c100 -d3, all 200.

 wrk -t1 -c100 -d3 http://remote:88/
Running 3s test @ http://remote:88/
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   142.11ms  228.44ms   1.79s    88.59%
    Req/Sec     1.00k   810.67     2.96k    83.33%
  2982 requests in 3.01s, 0.92MB read
  Socket errors: connect 0, read 0, write 0, timeout 3
Requests/sec:    989.16
Transfer/sec:    313.94KB

add (?i)(?: curl/) into /etc/nginx/waf/rules/user-agent


*   Trying 127.0.1.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55721df6cf90)
* Connected to localhost (127.0.1.1) port 88 (#0)
> GET / HTTP/1.1
> Host: localhost:88
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sun, 10 Jan 2021 04:33:31 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 2
< Last-Modified: Sun, 10 Jan 2021 04:08:30 GMT
< Connection: keep-alive
< Keep-Alive: timeout=120
< Vary: Accept-Encoding
< ETag: "5ffa7dbe-2"
< X-Content-Type-Options: nosniff
< Accept-Ranges: bytes
< 
a
* Connection #0 to host localhost left intact
* ```
ADD-SP commented 3 years ago

Do you have ngx_http_rewrite_module installed and using a configuration like the one below?

server {
    ....
    listen 88;
    waf on;
    waf mode STD;
    waf_rule_path /etc/nginx/waf/rules/;
    waf_cc_deny_limit 10 60;

    location / {
        return 200 'Hello World!';
    }
    ....
}

If so, please change it to something like this and retest it.

server {
    ....
    listen 88;
    waf on;
    waf mode STD;
    waf_rule_path /etc/nginx/waf/rules/;
    waf_cc_deny_limit 10 60;

    root /path/to/html;
    index index.html index.htm;
    ....
}

The reason for this is that ngx_http_rewrite_module is mounted in an earlier processing phase than ngx_waf, so if return is used it will return the result directly instead of calling a subsequent processing phase, and ngx_waf will not be able to inspect the request.

At the moment, no other directives has been found to have a similar effect.

phpstatic commented 3 years ago

I have no rewrite:

server {
        listen 88;
        root /var/www;
        include conf.d/waf.ini;
}

waf.ini

waf on;
waf_mode STD;
waf_rule_path /etc/nginx/waf/rules/;
waf_cc_deny_limit 500 60;

When I try this:

server {
        listen 88;
        root /var/www;
    waf on;
    waf mode STD;
    waf_rule_path /etc/nginx/waf/rules/;
    waf_cc_deny_limit 10 60;
}
nginx -t
nginx: [emerg] invalid number of arguments in "waf" directive in /etc/nginx/conf.d/aa.conf:5

In the log objs/addon/src/ngx_http_waf_module_core.o is link into nginx, now sure why has this error.

I always use include conf.d/wal.ini style to manage different module conf file, never see this kink error.

ADD-SP commented 3 years ago

Can you post the output of nginx -V?

It looks like you are using a static module, can you test the dynamic module to see if it has the same problem?

ADD-SP commented 3 years ago

I have no rewrite:

server {
        listen 88;
        root /var/www;
        include conf.d/waf.ini;
}

waf.ini

waf on;
waf_mode STD;
waf_rule_path /etc/nginx/waf/rules/;
waf_cc_deny_limit 500 60;

When I try this:

server {
        listen 88;
        root /var/www;
    waf on;
    waf mode STD;
    waf_rule_path /etc/nginx/waf/rules/;
    waf_cc_deny_limit 10 60;
}
nginx -t
nginx: [emerg] invalid number of arguments in "waf" directive in /etc/nginx/conf.d/aa.conf:5

In the log objs/addon/src/ngx_http_waf_module_core.o is link into nginx, now sure why has this error.

I always use include conf.d/wal.ini style to manage different module conf file, never see this kink error.

server {
        listen 88;
        root /var/www;
    waf on;
    waf mode STD;
    waf_rule_path /etc/nginx/waf/rules/;
    waf_cc_deny_limit 10 60;
}

There is a spelling error, waf mode should be corrected to waf_mode, there is an underscore between the two words.

phpstatic commented 3 years ago

Yes, the binary is static linked.

nginx -V
nginx version: nginx/1.18.0 (nginx)
built by gcc version 9.3.0
built with OpenSSL 1.1.0 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled
configure arguments:--conf-path=/etc/nginx/nginx.conf --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --user=www-data --group=www-data --without-select_module --with-poll_module --with-file-aio --with-threads --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_sub_module --with-http_stub_status_module --with-http_slice_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_secure_link_module --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-stream_realip_module --with-pcre --with-pcre-jit --with-openssl --with-zlib --with-zlib-asm=pentiumpro --with-libatomic --with-http_addition_module --without-http_uwsgi_module --with-mail --with-mail_ssl_module --with-http_mp4_module --with-http_flv_module --add-module=devel_kit --add-module=brotli --add-module=zstd --add-module=substitutions_filter --add-module=headers-more --add-module=h264_streaming --add-module=vod --add-module=secure-token --add-module=flv --add-module=dynamic_limit_req --add-module=slice --add-module=njs/nginx --add-module=vts --with-http_geoip_module --with-stream_geoip_module --add-module=geoip2 --add-module=ipip --add-module=ModSecurity --add-module=security_headers --add-module=tcp-keepalive --with-http_v3_module --with-openssl --with-quiche

change to waf_mode still get same results:

nginx -t
nginx: [emerg] invalid number of arguments in "waf" directive in /etc/nginx/conf.d/aa.conf:5
nginx: configuration file /etc/nginx/nginx.conf test failed
ADD-SP commented 3 years ago

Yes, the binary is static linked.

nginx -V
nginx version: nginx/1.18.0 (nginx)
built by gcc version 9.3.0
built with OpenSSL 1.1.0 (compatible; BoringSSL) (running with BoringSSL)
TLS SNI support enabled
configure arguments:--conf-path=/etc/nginx/nginx.conf --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --user=www-data --group=www-data --without-select_module --with-poll_module --with-file-aio --with-threads --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_sub_module --with-http_stub_status_module --with-http_slice_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_secure_link_module --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-stream_realip_module --with-pcre --with-pcre-jit --with-openssl --with-zlib --with-zlib-asm=pentiumpro --with-libatomic --with-http_addition_module --without-http_uwsgi_module --with-mail --with-mail_ssl_module --with-http_mp4_module --with-http_flv_module --add-module=devel_kit --add-module=brotli --add-module=zstd --add-module=substitutions_filter --add-module=headers-more --add-module=h264_streaming --add-module=vod --add-module=secure-token --add-module=flv --add-module=dynamic_limit_req --add-module=slice --add-module=njs/nginx --add-module=vts --with-http_geoip_module --with-stream_geoip_module --add-module=geoip2 --add-module=ipip --add-module=ModSecurity --add-module=security_headers --add-module=tcp-keepalive --with-http_v3_module --with-openssl --with-quiche

change to waf_mode still get same results:

nginx -t
nginx: [emerg] invalid number of arguments in "waf" directive in /etc/nginx/conf.d/aa.conf:5
nginx: configuration file /etc/nginx/nginx.conf test failed

Hello, I found some problems with the configure arguments you gave. I downloaded the source code from nginx.org. I didn't find the --with-http_v3_module and --with-quiche arguments, where did you download the source code from? Maybe we are using a different source code for NGINX.

ADD-SP commented 3 years ago

Hello, the latest stable version has been released, which supports printing debug logs. You can adjust the error log level to debug after the update and upload the error.log file to help locate the error, remember to clear the privacy information in the file.

phpstatic commented 3 years ago

The http3 patch is come from CloudFlare quiche patch. I will try and update the new version.

phpstatic commented 3 years ago

I can confirm the build is working now, but some time get Segmentation fault. not sure it is ngx_waf related.

ADD-SP commented 3 years ago

I can confirm the build is working now, but some time get Segmentation fault. not sure it is ngx_waf related.

You can compile a program suitable for debugging by changing the parameters -with-cc-opt='-O0 -g' and -with-debug in the configuration script. When you get an error, you can debug the core file with gdb and generally find out the location of the source code of the error.

ADD-SP commented 3 years ago

I closed this issue because I did not receive a response for more than 15 days, so you can reopen it if you have questions.