request help: How to import rule set files in Coraza waf

DokiDoki1103 commented 11 months ago

Issue description

see: https://github.com/apache/apisix-ingress-controller/issues/2043

I know the configuration method for a ruleset, but if multiple rulesets are referenced? Or is it possible to import files to make the firewall plugin effective? Configuration by configuration is too cumbersome.

This is the ruleset https://github.com/coreruleset/coreruleset/tree/v4.0/dev/rules

kayx23 commented 11 months ago

Try Include @owasp_crs/*.conf

DokiDoki1103 commented 11 months ago

Try Include @owasp_crs/*.conf

Hello sister, can you provide a complete demo? Also, which folder should these confs be placed in?

kayx23 commented 11 months ago

APISIX uses coraza-proxy-wasm, in which I found:

I have not tested this myself but I imagine updating configs here would serve the purpose:

Let me know if this works.

kayx23 commented 11 months ago

cc: @sn0rt @huanli-meng

DokiDoki1103 commented 11 months ago

Elder sister means that I need to import the firewall rules into a folder in apixsix, and then configure them directly like this?

curl -i http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "uri": "/anything",
  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "SecRule Include @owasp_crs/xxxx/xx.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}'

/xxxx/xx.conf is my path

kayx23 commented 11 months ago

The CRS should come embedded with the wasm module and you should not need to worry about the path.

The path is internal. owasp_crs gets mapped to crs so owasp_crs/*.conf correspond to these files that end with .conf: https://github.com/corazawaf/coraza-proxy-wasm/tree/main/wasmplugin/rules/crs

So I'd suggest you try this:

  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-demo-conf",
            "Include @owasp_crs/*.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  }

DokiDoki1103 commented 11 months ago

Dear sister, I still need your help

I configured yaml to enable firewall plugin support for a certain route, but it reported an error

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: httpbin-route
spec:
  http:
  - name: route-ip
    plugins:
    - name: coraza-filter
      enable: true
      config:
        default_directives: "default"
        directives_map:
          defalut: 
          - "SecDebugLogLevel 9"
          - "SecRuleEngine On"
          - "Include @crs-setup-demo-conf"
          - "Include @owasp_crs/*.conf"
    match:
      hosts:
      - test.lyck6.cn
      paths:
      - "/ip"
    upstreams:
    - name: httpbin-upstream

The error is because the plugin was not found

Events:
  Type     Reason               Age                   From           Message
  ----     ------               ----                  ----           -------
  Warning  ResourceSyncAborted  25s (x51 over 5m30s)  ApisixIngress  ApisixIngress synced failed, with error: 1 error occurred:
           * unknown plugin [coraza-filter]

Then I will configure configMap to support plugins, which is the document I referenced

https://apisix.apache.org/blog/2023/09/08/APISIX-integrates-with-Coraza/

There is a path that I don't know what I should fill in

/Home/ubuntu/coraza proxy wasm/build/main.wasm

The path I write by default does not work, do I still have any configuration errors?

DokiDoki1103 commented 11 months ago

I have read this document, but I still don't quite understand. What should I write in this path, or where should I find the file?

https://apisix.apache.org/zh/docs/apisix/wasm/

DokiDoki1103 commented 11 months ago

But I don't have this file /Home/ubuntu/coraza proxy wasm/build/main.wasm

kayx23 commented 11 months ago

Assuming the file path is correct. Have you reload APISIX after you update the config.yaml? Asking because the error message says unknown plugin [coraza-filter]

DokiDoki1103 commented 11 months ago

Should this be an external file? I don't have this document

kayx23 commented 11 months ago

Yes the wasm module needs to be downloaded first. Here's a doc based off the blog but offers more clarity in steps. It is based off an APISIX instance in Docker so just modify accordingly for your need.

DokiDoki1103 commented 11 months ago

Yes the wasm module needs to be downloaded first. Here's a doc based off the blog but offers more clarity in steps. It is based off an APISIX instance in Docker so just modify accordingly for your need.

I'll take a look at the documents first. Sister, please rest early. Thank you, sister

DokiDoki1103 commented 11 months ago

Sister, I'm sorry to bother you at night。

I followed the tutorial and moved the wasm file to my container. I also modified the configmap configuration file and restarted Apisix. However, when I configured it, I still prompted that the plugin was not found

The following is the version I am using

apache/apisix-ingress-controller:1.7.1
apache/apisix:3.6.0-centos

I came across this document. https://apisix.apache.org/zh/docs/apisix/wasm/

He said I need to build an apisix base environment, and I looked at this document again https://github.com/api7/apisix-build-tools

But when I build it myself, it will fail, so I used the Github action to build the already built image。

I replaced the image apache/apisix:3.6.0-centos -> api7/apisix-base:1.21.4.2.2

Then the container cannot start, I don't know if I want to directly replace the image .

DokiDoki1103 commented 11 months ago

我通过官网的最简单示例，成功配置单条规则，并且生效，但是我配置 Inlcude 规则，就会报错 500

我并没有将这些 conf 拷贝到容器里面，因为我不知道拷贝到哪个容器里面

这是日志

kayx23 commented 11 months ago

If you used the image we provide (e.g. apache/apisix:3.6.0-centos), you do not need to build APISIX Base as the APISIX provided is already on APISIX Base. You may safely ignore that note.

As for the issue with importing core rule sets, I'll take a look and get back to you.

kayx23 commented 11 months ago

The CRS should come embedded with the wasm module and you should not need to worry about the path.

The path is internal. owasp_crs gets mapped to crs so owasp_crs/*.conf correspond to these files that end with .conf: https://github.com/corazawaf/coraza-proxy-wasm/tree/main/wasmplugin/rules/crs

So I'd suggest you try this:
  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-demo-conf",
            "Include @owasp_crs/*.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  }

@DokiDoki1103 I can reproduce the error you see. You need to add "Include @crs-setup-demo-conf", as well, exactly as shown in my earlier comment. After that everything should work (I tested).

kayx23 commented 11 months ago

Just a caution note: once this PR (https://github.com/corazawaf/coraza-proxy-wasm/pull/243) is merged and if you use the new release in the future, you would update "Include @crs-setup-demo-conf" with Include @crs-setup-conf.

DokiDoki1103 commented 11 months ago

I tested in apisix quickstart and successfully configured it

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/*.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

My configuration has successfully worked

DokiDoki1103 commented 11 months ago

Then I tested a single rule and it didn't work

https://github.com/corazawaf/coraza-proxy-wasm/blob/main/wasmplugin/rules/crs/REQUEST-913-SCANNER-DETECTION.conf

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

curl -H "User-Agent: urlgrabber/1.0" http://120.26.1.38:9080/anything/1

It did not intercept malicious requests and return 200

this is log

124.42.51.91 - - [17/Nov/2023:01:42:39 +0000] lyck6.cn "GET /anything/1 HTTP/1.1" 200 435 0.423 "-" "urlgrabber/1.0" 35.153.249.234:80 200 0.415 "http://lyck6.cn"
2023/11/17 01:42:46 [error] 239#239: *3592185 Invalid value tx_id="ScUeUyFLLIeDmStPISf" var_value="+tx.critical_anomaly_score" rule_id=0 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "GET /anything/1 HTTP/1.1", host: "lyck6.cn"
2023/11/17 01:42:46 [emerg] 239#239: *3592185 [client ""] Coraza: Warning. Found User-Agent associated with security scanner [file "@owasp_crs/REQUEST-913-SCANNER-DETECTION.conf"] [line "786"] [id "913100"] [rev ""] [msg "Found User-Agent associated with security scanner"] [data "Matched Data: grabber found within MATCHED_VARS:REQUEST_HEADERS:user-agent: urlgrabber/1.0"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scanner"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/118/224/541/310"] [tag "PCI/6.5.10"] [hostname ""] [uri "/anything/1"] [unique_id "ScUeUyFLLIeDmStPISf"], client: 124.42.51.91, server: _, request: "GET /anything/1 HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:01:42:46 +0000] lyck6.cn "GET /anything/1 HTTP/1.1" 200 435 0.740 "-" "urlgrabber/1.0" 54.161.176.214:80 200 0.741 "http://lyck6.cn"

kayx23 commented 11 months ago

See this https://groups.google.com/a/owasp.org/g/modsecurity-core-rule-set-project/c/rFeNiHGAFdg

DokiDoki1103 commented 11 months ago

I set it but it doesn't take effect ： "SecDefaultAction phase:1,log,auditlog,deny,status:403",

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "SecDefaultAction phase:1,log,auditlog,deny,status:403",
                        "SecDefaultAction phase:2,log,auditlog,deny,status:403",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf",
                        "Include @owasp_crs/REQUEST-921-PROTOCOL-ATTACK.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

phase: phase:1,log,auditlog,pass
2023/11/17 03:31:22 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:22 [error] 237#237: *4015032 [lua] wasm.lua:115: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:03:31:21 +0000] 120.26.1.38:9180 "PUT /apisix/admin/routes/ HTTP/1.1" 200 696 0.004 "-" "PostmanRuntime/7.34.0" - - - "http://120.26.1.38:9180"
124.42.51.91 - - [17/Nov/2023:03:31:22 +0000] lyck6.cn "POST /anything/unix HTTP/1.1" 503 269 0.000 "-" "urlgrabber/1.0" - - - "http://lyck6.cn"
2023/11/17 03:31:29 [emerg] 237#237: Failed to parse directives: invalid WAF config from string: failed to parse string: failed to compile the directive "secaction": SecDefaultAction already defined for this phase: phase:1,log,auditlog,pass
2023/11/17 03:31:29 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:29 [error] 237#237: *4015032 [lua] wasm.lua:79: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:31:29 [warn] 237#237: *4015032 [lua] plugin.lua:1107: run_plugin(): coraza-filter exits with http status code 503, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:31:29 [emerg] 237#237: Failed to parse directives: invalid WAF config from string: failed to parse string: failed to compile the directive "secaction": SecDefaultAction already defined for this phase: phase:1,log,auditlog,pass
2023/11/17 03:31:29 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:29 [error] 237#237: *4015032 [lua] wasm.lua:115: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:03:31:29 +0000] lyck6.cn "POST /anything/unix HTTP/1.1" 503 269 0.000 "-" "PostmanRuntime/7.34.0" - - - "http://lyck6.cn"

kayx23 commented 11 months ago

The specific issue isn't with APISIX anymore as the integration has now worked as intended.

For questions about specific rules, please redirect them to coraza-proxy-wasm or the relevant coraza communication channels. Thanks.

DokiDoki1103 commented 11 months ago

I tested that not only one plugin is not working, it seems that all plugins are not working, such as

https://github.com/coreruleset/coreruleset/blob/v3.2/dev/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",

                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941160 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941160 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. NoScript XSS InjectionChecker: HTML Injection [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "943"] [id "941160"] [rev ""] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: <body  found within REQUEST_COOKIES_NAMES:<body onload: <body onload"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941390 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941390 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. Javascript method detected [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "1437"] [id "941390"] [rev ""] [msg "Javascript method detected"] [data "Matched Data: alert( found within REQUEST_COOKIES:<body onload: \"alert(1)\">"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941320 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941320 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. Possible XSS Attack Detected - HTML Tag Handler [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "1668"] [id "941320"] [rev ""] [msg "Possible XSS Attack Detected - HTML Tag Handler"] [data "Matched Data: <body  found within REQUEST_COOKIES_NAMES:<body onload: <body onload"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "OWASP_CRS"] [tag "capec/1000/152/242/63"] [tag "PCI/6.5.1"] [tag "paranoia-level/2"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"

DokiDoki1103 commented 11 months ago

The problem is that I can refer to * to make all rule sets effective, but referring to a single rule set does not

kayx23 commented 11 months ago

The log you provided showed the single rule was effective and Coraza was detecting. It just wasn't blocking. This isn't an APISIX issue per se. Please do more research on your end.

DokiDoki1103 commented 11 months ago

Dear sister, after continuous attempts, I have also successfully configured a single rule。

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-901-INITIALIZATION.conf",
                        "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf",
                        "Include @owasp_crs/REQUEST-949-BLOCKING-EVALUATION.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

Configuring rule 913 in this way will succeed

901 must be added before this
949 must be added after this

That is to say, if one's own rules need to be placed in the middle, they must ensure consistency in order. I tested that if the order is incorrect, it will not take effect

kayx23 commented 11 months ago

Ok great, thanks for sharing that info. I've tracked this and at some point we'll have a doc update to provide more examples, especially for including the CRS.

If everything works out well now, we can close the issue?

DokiDoki1103 commented 11 months ago

Yes, this issue can be resolved, but I encountered another issue when using apisix ingress controller. He will prompt me that I haven't found the plugin

https://github.com/apache/apisix-ingress-controller/issues/2049

apache / apisix-ingress-controller

request help: How to import rule set files in Coraza waf #2044

Issue description