How to use CRS correctly to have basic security protection

[naiba]

👋 I am currently making a coraza-waf extension for https://github.com/caddyserver/caddy, which has been basically completed, but found that all requests after the phase2 process failed to pass the inspection.

How should I enable CRS correctly?

c.ctx = ctx
c.logger = ctx.Logger(c)
defer c.logger.Sync()

c.Waf = engine.NewWaf()
c.Waf.Init()
c.Waf.InitLogger()

crs, err := crs.NewCrs(c.Waf)
if err != nil {
    return err
}
crs.TemplateDir = "/home/naiba/Desktop/coraza-waf/docs/crs/rules"
err = crs.Build()
if err != nil {
    return err
}
l := len(c.Waf.Rules.GetRules())

if l == 0 {
    return errors.New("No rules found")
}

if l < 500 {
    return fmt.Errorf("Not enough CRS rules, found %d", l)
}

func (h *HTTPHandler) ServeHTTP(w http.ResponseWriter, r *http.Request, next caddyhttp.Handler) error {
    tx := h.Waf.NewTransaction()

    err := tx.ParseRequestObjectHeaders(r)
    if err != nil {
        h.logger.Error("ParseRequestObjectHeaders", zap.Any("err", err))
        return err
    }
    if tx.ExecutePhase(1) {
        w.Write([]byte(tx.GetErrorPage()))
        return errors.New("phase1")
    }

    err = tx.ParseRequestObjectBody(r)
    if err != nil {
        h.logger.Error("ParseRequestObjectBody", zap.Any("err", err))
        return err
    }
    if tx.ExecutePhase(2) {
        h.logger.Info("MatchedRules", zap.Any("disrupted", tx.Disrupted), zap.Any("rules", tx.MatchedRules))
        w.Write([]byte(tx.GetErrorPage()))
        return errors.New("phase2")
    }

    // Continue down the handler stack
    if err := next.ServeHTTP(w, r); err != nil {
        return err
    }

    return nil
}

[jptosso]

Hello @naiba, thank you for building a plugin with Coraza WAF, I really hope it can be the foundation for many projects.

Regarding your issue, the problem probably isn't Coraza WAF itself but OWASP CRS, please check this file: https://github.com/jptosso/coraza-waf/blob/master/pkg/crs/crs.go and try to setup the CRS struct with valid values, here is an example of a working implementation: https://github.com/jptosso/coraza-waf/blob/66c4d34c2930752d198f97c8fdf8722f1664f1df/examples/rpc/service.yaml

The idea of the CRS engine is to create a CRS profile with settings, using the default values might fail.

If you have any further issues please submit a debug log with detailed information about the triggered rules.

Regards

[naiba]

Hello, for the information you provided, I did some tests and the result is

phase1 passed
phase2 passed
phase4 failed
phase5 failed

I am sure that the configuration of WAF and CRS has been deserialized into the object correctly, but I have three confusions:

1. profile: For this configuration item, I directly use the configuration of examples/skipper/default.conf.
2. config.geoipdb: I commented out this configuration item because no open data was found
3. config. unicode_map: I commented out this configuration too, did not understand the meaning of this configuration item, and did not find a sample configuration

Is it because the above three configuration errors cause all requests to fail?

[jptosso]

Try printing tx.DisruptiveRuleId after fail, after that try disabling:

    response_body_access: false
    body_inspection: false

Profile and geoip are fine, unicode_map is used to avoid false positives while processing unicode characters, for example, you may set a unicode mapping for japanese kanji, hiragana and katakana, so greek characters won't pass.

[naiba]

Feeling that we are just one step away from success, I think the CRS rule is working now, but it still fails the inspection for some reason, I printed the log:

INFO    http.handlers.waf   Disruptived {"DisruptiveRuleId": 949110, "Msg": ["", "Previous Block Reason: ", "GET", "GET", "HTTP/1.1", "Restricted header detected: cache-control"]}

Then I commented out the rule Restricted header detected, but it still failed the inspection:

INFO    http.handlers.waf   Disruptived {"DisruptiveRuleId": 949110, "Msg": ["", "Previous Block Reason: ", "GET", "GET", "HTTP/1.1"]}

Finally, I modified the WAF configuration file according to your suggestions

response_body_access: false
body_inspection: false

and got the following results

INFO    http.handlers.waf   Disruptived {"DisruptiveRuleId": 949110, "Msg": ["", "Previous Block Reason: ", "GET", "GET", "HTTP/1.1"]}

[jptosso]

Rule 949110 uses TX:ANOMALY_SCORE and compares it with inbound_anomaly_score_threshold

Could you debug the TX object? it would be tx.Collections.Data["tx"]

[naiba]

Hi, there is a full log. The score does not seem to exceed the threshold?

{
    "DisruptiveRuleId": 949110,
    "Msg": [
        "",
        "Previous Block Reason: ",
        "GET",
        "GET",
        "HTTP/1.1"
    ],
    "Collections": {
        "appid": {
            "data": {
                "": [

                ]
            },
            "Name": "appid"
        },
        "args": {
            "data": {
                "": [

                ]
            },
            "Name": "args"
        },
        "args_get": {
            "data": {
                "": [

                ]
            },
            "Name": "args_get"
        },
        "args_get_names": {
            "data": {
                "": [

                ]
            },
            "Name": "args_get_names"
        },
        "args_names": {
            "data": {
                "": [

                ]
            },
            "Name": "args_names"
        },
        "args_post": {
            "data": {
                "": [

                ]
            },
            "Name": "args_post"
        },
        "args_post_names": {
            "data": {
                "": [

                ]
            },
            "Name": "args_post_names"
        },
        "files": {
            "data": {
                "": [

                ]
            },
            "Name": "files"
        },
        "files_combined_size": {
            "data": {
                "": [

                ]
            },
            "Name": "files_combined_size"
        },
        "files_names": {
            "data": {
                "": [

                ]
            },
            "Name": "files_names"
        },
        "files_sizes": {
            "data": {
                "": [

                ]
            },
            "Name": "files_sizes"
        },
        "full_request": {
            "data": {
                "": [

                ]
            },
            "Name": "full_request"
        },
        "global": {
            "data": {
                "CREATE_TIME": [
                    "1613309098799061238"
                ],
                "IS_NEW": [
                    "1"
                ],
                "LAST_UPDATE_TIME": [
                    "1613309098817254689"
                ],
                "TIMEOUT": [
                    "0"
                ],
                "UPDATE_COUNTER": [
                    "1"
                ],
                "UPDATE_RATE": [
                    "0"
                ]
            },
            "Name": "global"
        },
        "id": {
            "data": {
                "": [
                    "DWA6B61Z17RNYSRD8XT"
                ]
            },
            "Name": "id"
        },
        "ip": {
            "data": {
                "CREATE_TIME": [
                    "1613309098799070450"
                ],
                "IS_NEW": [
                    "1"
                ],
                "LAST_UPDATE_TIME": [
                    "1613309098817286639"
                ],
                "TIMEOUT": [
                    "0"
                ],
                "UPDATE_COUNTER": [
                    "1"
                ],
                "UPDATE_RATE": [
                    "0"
                ],
                "dos_block": [
                    "1"
                ],
                "dos_burst_counter": [
                    "2"
                ]
            },
            "Name": "ip"
        },
        "matched_var": {
            "data": {
                "": [
                    ""
                ]
            },
            "Name": "matched_var"
        },
        "matched_var_name": {
            "data": {
                "": [
                    ""
                ]
            },
            "Name": "matched_var_name"
        },
        "matched_vars": {
            "data": {
                "": [
                    ":"
                ]
            },
            "Name": "matched_vars"
        },
        "query_string": {
            "data": {
                "": [
                    ""
                ]
            },
            "Name": "query_string"
        },
        "remote_addr": {
            "data": {
                "": [
                    ""
                ]
            },
            "Name": "remote_addr"
        },
        "remote_host": {
            "data": {
                "": [

                ]
            },
            "Name": "remote_host"
        },
        "remote_port": {
            "data": {
                "": [
                    "0"
                ]
            },
            "Name": "remote_port"
        },
        "remote_user": {
            "data": {
                "": [

                ]
            },
            "Name": "remote_user"
        },
        "reqbody_processor": {
            "data": {
                "": [

                ]
            },
            "Name": "reqbody_processor"
        },
        "request_basename": {
            "data": {
                "": [
                    ""
                ]
            },
            "Name": "request_basename"
        },
        "request_body": {
            "data": {
                "": [

                ]
            },
            "Name": "request_body"
        },
        "request_body_length": {
            "data": {
                "": [
                    "0"
                ]
            },
            "Name": "request_body_length"
        },
        "request_content_length": {
            "data": {
                "": [

                ]
            },
            "Name": "request_content_length"
        },
        "request_content_type": {
            "data": {
                "": [

                ]
            },
            "Name": "request_content_type"
        },
        "request_cookies": {
            "data": {
                "": [

                ]
            },
            "Name": "request_cookies"
        },
        "request_cookies_names": {
            "data": {
                "": [

                ]
            },
            "Name": "request_cookies_names"
        },
        "request_filename": {
            "data": {
                "": [
                    "/"
                ]
            },
            "Name": "request_filename"
        },
        "request_headers": {
            "data": {
                "": [

                ],
                "accept": [
                    "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
                ],
                "accept-encoding": [
                    "gzip, deflate"
                ],
                "accept-language": [
                    "en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7"
                ],
                "connection": [
                    "keep-alive"
                ],
                "content-length": [
                    "0"
                ],
                "dnt": [
                    "1"
                ],
                "upgrade-insecure-requests": [
                    "1"
                ],
                "user-agent": [
                    "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36"
                ]
            },
            "Name": "request_headers"
        },
        "request_headers_names": {
            "data": {
                "": [
                    "connection",
                    "dnt",
                    "upgrade-insecure-requests",
                    "user-agent",
                    "accept",
                    "accept-encoding",
                    "accept-language"
                ]
            },
            "Name": "request_headers_names"
        },
        "request_line": {
            "data": {
                "": [
                    "GET / HTTP/1.1"
                ]
            },
            "Name": "request_line"
        },
        "request_method": {
            "data": {
                "": [
                    "GET"
                ]
            },
            "Name": "request_method"
        },
        "request_protocol": {
            "data": {
                "": [
                    "HTTP/1.1"
                ]
            },
            "Name": "request_protocol"
        },
        "request_uri": {
            "data": {
                "": [
                    "/",
                    "/"
                ]
            },
            "Name": "request_uri"
        },
        "request_uri_raw": {
            "data": {
                "": [
                    "/"
                ]
            },
            "Name": "request_uri_raw"
        },
        "response_body": {
            "data": {
                "": [

                ]
            },
            "Name": "response_body"
        },
        "response_content_length": {
            "data": {
                "": [

                ]
            },
            "Name": "response_content_length"
        },
        "response_content_type": {
            "data": {
                "": [

                ]
            },
            "Name": "response_content_type"
        },
        "response_headers": {
            "data": {
                "": [

                ]
            },
            "Name": "response_headers"
        },
        "response_headers_names": {
            "data": {
                "": [

                ]
            },
            "Name": "response_headers_names"
        },
        "response_protocol": {
            "data": {
                "": [

                ]
            },
            "Name": "response_protocol"
        },
        "response_status": {
            "data": {
                "": [

                ]
            },
            "Name": "response_status"
        },
        "rule": {
            "data": {
                "": [

                ],
                "id": [
                    "0"
                ],
                "msg": [
                    ""
                ],
                "rev": [
                    ""
                ],
                "severity": [
                    ""
                ]
            },
            "Name": "rule"
        },
        "session": {
            "data": {
                "": [

                ]
            },
            "Name": "session"
        },
        "timestamp": {
            "data": {
                "": [
                    "1613309098797670677"
                ]
            },
            "Name": "timestamp"
        },
        "tx": {
            "data": {
                "": [

                ],
                "0": [

                ],
                "1": [

                ],
                "10": [

                ],
                "2": [

                ],
                "3": [

                ],
                "4": [

                ],
                "5": [

                ],
                "6": [

                ],
                "7": [

                ],
                "8": [

                ],
                "9": [

                ],
                "allowed_http_versions": [
                    "HTTP/1.0HTTP/1.1HTTP/2HTTP/2.0"
                ],
                "allowed_methods": [
                    "GETHEADPOSTOPTIONS"
                ],
                "allowed_request_content_type": [
                    "|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/cloudevents+json| |application/cloudevents-batch+json| |application/octet-stream| |application/csp-report| |application/xss-auditor-report| |text/plain|"
                ],
                "allowed_request_content_type_charset": [
                    "utf-8|iso-8859-1|iso-8859-15|windows-1252"
                ],
                "anomaly_score": [
                    "8"
                ],
                "anomaly_score_pl1": [
                    "8"
                ],
                "anomaly_score_pl2": [
                    "0"
                ],
                "anomaly_score_pl3": [
                    "0"
                ],
                "anomaly_score_pl4": [
                    "0"
                ],
                "arg_length": [
                    "400"
                ],
                "arg_name_length": [
                    "100"
                ],
                "combined_file_sizes": [
                    "5048576"
                ],
                "critical_anomaly_score": [
                    "5"
                ],
                "crs_setup_version": [
                    "300"
                ],
                "crs_validate_utf8_encoding": [
                    "0"
                ],
                "do_reput_block": [
                    "1"
                ],
                "dos_block_timeout": [
                    "600"
                ],
                "dos_burst_time_slice": [
                    "60"
                ],
                "dos_counter_threshold": [
                    "100"
                ],
                "enforce_bodyproc_urlencoded": [
                    "1"
                ],
                "error_anomaly_score": [
                    "4"
                ],
                "executing_paranoia_level": [
                    "1"
                ],
                "extension": [
                    "//"
                ],
                "high_risk_country_codes": [
                    ""
                ],
                "http_violation_score": [
                    "0"
                ],
                "inbound_anomaly_score": [
                    "8"
                ],
                "inbound_anomaly_score_threshold": [
                    "100"
                ],
                "ip_whitelist": [
                    "127.0.0.1/32"
                ],
                "lfi_score": [
                    "0"
                ],
                "max_file_size": [
                    "0"
                ],
                "max_num_args": [
                    "255"
                ],
                "notice_anomaly_score": [
                    "2"
                ],
                "outbound_anomaly_score": [
                    "0"
                ],
                "outbound_anomaly_score_pl1": [
                    "0"
                ],
                "outbound_anomaly_score_pl2": [
                    "0"
                ],
                "outbound_anomaly_score_pl3": [
                    "0"
                ],
                "outbound_anomaly_score_pl4": [
                    "0"
                ],
                "outbound_anomaly_score_threshold": [
                    "100"
                ],
                "paranoia_level": [
                    "1"
                ],
                "php_injection_score": [
                    "0"
                ],
                "rce_score": [
                    "0"
                ],
                "real_ip": [
                    ""
                ],
                "reput_block_duration": [
                    "300"
                ],
                "restricted_extensions": [
                    ".backup/.bak/.bat/.cdx/.cer/.cfg/.cmd/.com/.config/.conf/.cs/.csproj/.csr/.dat/.db/.dbf/.dll/.dos/.htr/.htw/.ida/.idc/.idq/.inc/.ini/.key/.licx/.lnk/.log/.mdb/.old/.pass/.pdb/.pol/.printer/.pwd/.rdb/.resources/.resx/.sql/.swp/.sys/.vb/.vbs/.vbproj/.vsdisco/.webinfo/.xsd/.xsx/.config/.conf/.db/.ini/.log/.old/.pass/.pdb/.rdb/.sql"
                ],
                "restricted_headers": [
                    "/proxy/ /lock-token/ /content-range/ /if/"
                ],
                "rfi_score": [
                    "0"
                ],
                "sampling_percentage": [
                    "100"
                ],
                "session_fixation_score": [
                    "0"
                ],
                "sql_error_match": [
                    "0"
                ],
                "sql_injection_score": [
                    "0"
                ],
                "static_extensions": [
                    "/.jpg/ /.jpeg/ /.png/ /.gif/ /.js/ /.css/ /.ico/ /.svg/ /.webp/"
                ],
                "total_arg_length": [
                    "64000"
                ],
                "ua_hash": [
                    "2d0d3a5d5f93200793530b9ca2a9edafa835f205"
                ],
                "warning_anomaly_score": [
                    "3"
                ],
                "xss_score": [
                    "0"
                ]
            },
            "Name": "tx"
        },
        "xml": {
            "data": {
                "": [

                ]
            },
            "Name": "xml"
        }
    }
}

[jptosso]

Fixed, @ge wasn't taking macro expansions, going to be releasing patch today.

[naiba]

Thank you, I will come back after a few days to try again.