p0pr0ck5 / lua-resty-waf

High-performance WAF built on the OpenResty stack
GNU General Public License v3.0
1.28k stars 305 forks source link

Hints for install verification? #288

Closed kwaping closed 7 years ago

kwaping commented 7 years ago

I think I have this software properly installed but I can't get it to deny any requests. I'm looking through the included rulesets and have been crafting curl requests that I expect to trigger them. So far, I haven't had a single one get denied (and no log messages either).

Do I have to load the default rules explicitly, or is that done automatically? If that is done automatically, do you have any pointers about how I can verify that everything is working properly?

Thanks in advance!

kwaping commented 7 years ago

Here's an example of a test I tried:

curl -v -H "User-Agent: WinHttp.WinHttpRequest.5"  http://localhost/
* About to connect() to localhost port 80 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> Accept: */*
> User-Agent: WinHttp.WinHttpRequest.5
> 
< HTTP/1.1 200 OK
< Date: Mon, 28 Aug 2017 19:36:59 GMT
< Content-Type: text/html
< Content-Length: 562
< Connection: keep-alive
< Last-Modified: Tue, 18 Jul 2017 12:57:16 GMT
< ETag: "596e05ac-232"
< Accept-Ranges: bytes
< 
<!DOCTYPE html>
<html>
<head>
<title>Welcome to OpenResty!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to OpenResty!</h1>
<p>If you see this page, the OpenResty web platform is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="https://openresty.org/">openresty.org</a>.<br/></p>

<p><em>Thank you for flying OpenResty.</em></p>
</body>
</html>
* Connection #0 to host localhost left intact

The logs (proxy with WAF on 80, the destination is on 8080):

==> /var/log/nginx/access.log <==
127.0.0.1 - - [28/Aug/2017:19:36:59 +0000] "GET / HTTP/1.0" 200 562 "-" "WinHttp.WinHttpRequest.5" "-" "-" "-" "-" - 0.000 failover:-
127.0.0.1 - - [28/Aug/2017:19:36:59 +0000] "GET / HTTP/1.1" 200 562 "-" "WinHttp.WinHttpRequest.5" "-" "127.0.0.1:8080" "-" "-" 0.001 0.001 failover:-

And my Nginx configs:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
# MANAGED BY PUPPET

### snip ###

http {
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;
  log_format main '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" "$upstream_addr" "$ssl_cipher" "$upstream_cache_status" $upstream_response_time $request_time failover:$http_x_proofpoint_failover';

  access_log  /var/log/nginx/access.log;

  sendfile    on;
  tcp_nopush on;
  server_tokens off;

  types_hash_max_size 1024;
  types_hash_bucket_size 512;

  server_names_hash_bucket_size 64;
  server_names_hash_max_size 512;

  keepalive_timeout   360;
  keepalive_requests  32768;
  client_body_timeout 60;
  send_timeout        60;
  lingering_timeout   5;
  tcp_nodelay         on;

  gzip              on;
  gzip_comp_level   1;
  gzip_disable      msie6;
  gzip_min_length   20;
  gzip_http_version 1.1;
  gzip_proxied      off;
  gzip_vary         off;

  client_body_temp_path   /var/nginx/client_body_temp;
  client_max_body_size    10m;
  client_body_buffer_size 128k;
  proxy_temp_path         /var/nginx/proxy_temp;
  proxy_connect_timeout   90;
  proxy_send_timeout      90;
  proxy_read_timeout      90;
  proxy_buffers           32 4k;
  proxy_buffer_size       8k;
  proxy_set_header        Host $host;
  proxy_set_header        X-Real-IP $remote_addr;
  proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header        Proxy "";
  proxy_headers_hash_bucket_size 64;

  more_clear_headers Server:*;

  include /etc/nginx/conf.d/*.conf;
  include /etc/nginx/sites-enabled/*;
}

# configuration file /etc/nginx/mime.types:
# MANAGED BY PUPPET
### snip ###

# configuration file /etc/nginx/conf.d/lua-resty-waf_init.conf:
# needed by lua-resty-waf module config

init_by_lua_block {
    require "resty.core"
    local lua_resty_waf = require "resty.waf"
    lua_resty_waf.init()
}

# configuration file /etc/nginx/sites-enabled/m0077569.lab.ppops.net.conf:
# MANAGED BY PUPPET
server {
  listen *:80 reuseport deferred;
  server_name           ### redacted ###;

  resolver                   ### redacted ###;
  index  index.html index.htm index.php;

  access_log            /var/log/nginx/access.log main;
  error_log             /var/log/nginx/### redacted ###.error.log;

  location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"
        local waf = lua_resty_waf:new()
        waf:set_option("debug", true)
        waf:set_option("mode", "ACTIVE")
        waf:set_option("event_log_periodic_flush", 5)
        waf:exec()
    }

    header_filter_by_lua_block {
        local lua_resty_waf = require "resty.waf"
        local waf = lua_resty_waf:new()
        waf:exec()
    }

    body_filter_by_lua_block {
        local lua_resty_waf = require "resty.waf"
        local waf = lua_resty_waf:new()
        waf:exec()
    }

    log_by_lua_block {
        local lua_resty_waf = require "resty.waf"
        local waf = lua_resty_waf:new()
        waf:write_log_events()
        waf:exec()
    }
    proxy_pass            $scheme://localhost:8080;
    proxy_read_timeout    90;
    proxy_connect_timeout 90;
    proxy_set_header      Host $host;
  }
}

# configuration file /etc/nginx/sites-enabled/### redacted ###:8080.conf:
# MANAGED BY PUPPET
server {
  listen *:8080 reuseport deferred;
  server_name           ### redacted ###:8080;

  resolver                   ### redacted ###;
  index  index.html index.htm index.php;

  access_log            /var/log/nginx/access.log main;
  error_log             /var/log/nginx/### redacted ###:8080.error.log;

  location /test {
    index     index.html index.htm index.php;
    default_type text/plain;
    return 200 "Thank you for requesting ${request_uri}\n";
  }

  location / {
    root      /var/www/html/;
    index     index.html index.htm index.php;
  }
}
p0pr0ck5 commented 7 years ago

Hey @kwaping, as you have waf:set_option("debug", true), you should be able to walk through the error.log to examine the behavior in play. Make sure that the log level is set properly (e.g., https://github.com/p0pr0ck5/lua-resty-waf#debug_log_level; by default Nginx error_log level is NOTICE).

p0pr0ck5 commented 7 years ago

Also, off the top of my head I suspect the above request isn't enough to trip the score_threshold value. Maybe try something a bit more sinister? ;)

curl -v http://localhost/?foo=<script>

kwaping commented 7 years ago

Cool, thanks! I will take your suggestions and see how it goes.

kwaping commented 7 years ago

Is there documentation of the rule fields and format somewhere? I'm new to this and taking stabs at what some of these things mean.

p0pr0ck5 commented 7 years ago

Can you clarify your meaning behind "rule fields and format"? Do you mean the WAF rule definitions? If so, those values are translated from ModSecurity rules, so I would suggest reading up on the SecRules DSL. Otherwise, let me know to what you refer :)

kwaping commented 7 years ago

Yeah, that's what I meant, the fields and format inside the default rulesets that are provided such as 40000_generic_attack.json. Thanks!

p0pr0ck5 commented 7 years ago

Since they are automatically generated there is no direct documentation. Possibly something to work on as we move toward CRS v3 integration. :)

kwaping commented 7 years ago

Still not getting any hits. I added the debug setting you suggested, plus waf:set_option("score_threshold", 1) hoping that would lower it enough. Do you have an actual URI or query string that I could copy that will definitely trigger a hit?

p0pr0ck5 commented 7 years ago

I noted above :) :

curl -v http://localhost/?foo=<script>

what does the contents of your error.log look like? and did you check your error log debug level?

kwaping commented 7 years ago

Oh, didn't realize that was literal! I thought <script> was a fill-in-the-blank that I had to craft myself. I'll check the error log debug level, thanks.

kwaping commented 7 years ago

Excellent!

# curl -v  'http://localhost/?foo=<script>'
* About to connect() to localhost port 80 (#0)
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /?foo=<script> HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< Date: Mon, 28 Aug 2017 21:30:32 GMT
< Content-Type: text/html
< Content-Length: 162
< Connection: keep-alive
< 
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host localhost left intact

Logs:

127.0.0.1 - - [28/Aug/2017:21:30:32 +0000] "GET /?foo=<script> HTTP/1.1" 403 162 "-" "curl/7.29.0" "-" "-" "-" "-" - 0.000 failover:-