search

corazawaf / coraza

OWASP Coraza WAF is a golang modsecurity compatible web application firewall library
https://www.coraza.io
Apache License 2.0
2.25k stars 222 forks source link

AuditLog parity with libmodsecurity3 #856

Open amsnek opened 1 year ago

amsnek commented 1 year ago

Summary

Corazas current AuditLog format in json has empty values for important fields and is overall less detailed compared to libmodsecurity3. Since the stated goal is to be a "drop in replacement" for modsecurity, logging parity would be beneficial.

Basic example

Below are AuditLogs from coraza-spoa ccompared to libmodsecurity3 with the following http request intended to trigger the WAF. curl -v http://127.0.0.100/?x\=/etc/passwd

AuditLog generated from coraza-spoa 

{
  "transaction": {
    "timestamp": "2023/08/05 13:58:06",
    "unix_timestamp": 1691236686919190605,
    "id": "622c3c2b-073f-43c8-9878-657be5d8a69f",
    "client_ip": "127.0.0.1",
    "client_port": 6203,
    "host_ip": "127.0.0.100",
    "host_port": 80,
    "server_id": "",
    "request": {
      "method": "",
      "protocol": "",
      "uri": "",
      "http_version": "",
      "headers": {
        "accept": [
          "*/*"
        ],
        "host": [
          "127.0.0.100"
        ],
        "user-agent": [
          "curl/7.61.1"
        ]
      },
      "body": "",
      "files": null
    },
    "response": {
      "protocol": "",
      "status": 0,
      "headers": {},
      "body": ""
    },
    "producer": {
      "connector": "",
      "version": "",
      "server": "",
      "rule_engine": "On",
      "stopwatch": "1691236686919190605 1091256; combined=1046168, p1=360327, p2=646155, p3=0, p4=0, p5=39686",
      "rulesets": [
        "OWASP_CRS/4.0.0-rc1"
      ]
    }
  }
}

There are details in coraza-spoa default error log but this is only partially json and not compareable to the auditlog:

{
  "level": "error",
  "ts": 1691236686.9202244,
  "msg": "[client \"127.0.0.1\"] Coraza: Access denied (phase 2). Inbound Anomaly Score Exceeded (Total Score: 13) [file \"/path/to/rules/REQUEST-949-BLOCKING-EVALUATION.conf\"] [line \"11001\"] [id \"949110\"] [rev \"\"] [msg \"Inbound Anomaly Score Exceeded (Total Score: 13)\"] [data \"\"] [severity \"emergency\"] [ver \"OWASP_CRS/4.0.0-rc1\"] [maturity \"0\"] [accuracy \"0\"] [tag \"anomaly-evaluation\"] [hostname \"127.0.0.100\"] [uri \"/?x=/etc/passwd\"] [unique_id \"622c3c2b-073f-43c8-9878-657be5d8a69f\"]\n"
}

AuditLog from libmodsecurity3

  "transaction": {
    "client_ip": "127.0.0.1",
    "time_stamp": "Thu Aug  3 22:42:14 2023",
    "server_id": "5dab6b80f157",
    "client_port": 0,
    "host_ip": "127.0.0.100",
    "host_port": 80,
    "unique_id": "d181c234abcd1be01421",
    "request": {
      "method": "GET",
      "http_version": 1.1,
      "uri": "/?x=/etc/passwd",
      "body": "",
      "headers": {
        "host": "example.tld",
        "user-agent": "curl/7.61.1",
        "accept": "*/*",
        "x-forwarded-for": "127.0.0.1"
      }
    },
    "response": {
      "body": "<!doctype html>\n<head><meta charset=\"utf-8\">\n<title>403 Forbidden</title>\n</head>\n<style> .... ",
      "http_code": 403,
      "headers": {
        "Server": "nginx",
        "Date": "Thu, 03 Aug 2023 20:42:14 GMT",
        "Content-Type": "text/html",
        "Connection": "keep-alive"
      }
    },
    "producer": {
      "modsecurity": "ModSecurity v3.0.10 (Linux)",
      "connector": "ModSecurity-nginx v1.0.3",
      "secrules_engine": "Enabled",
      "components": [
        "OWASP_CRS/3.3.5\""
      ]
    },
    "messages": [
      {
        "message": "OS File Access Attempt",
        "details": {
          "match": "Matched \"Operator `PmFromFile' with parameter `lfi-os-files.data' against variable `ARGS:x' (Value: `/etc/passwd' )",
          "reference": "o1,10v8,11t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercase",
          "ruleId": "930120",
          "file": "/path/to/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf",
          "lineNumber": "79",
          "data": "Matched Data: etc/passwd found within ARGS:x: /etc/passwd",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-lfi",
            "paranoia-level/1",
            "OWASP_CRS",
            "capec/1000/255/153/126",
            "PCI/6.5.4"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      },
      {
        "message": "Remote Command Execution: Unix Shell Code Found",
        "details": {
          "match": "Matched \"Operator `PmFromFile' with parameter `unix-shell.data' against variable `ARGS:x' (Value: `/etc/passwd' )",
          "reference": "o1,10v8,11t:urlDecodeUni,t:cmdLine,t:normalizePath,t:lowercase",
          "ruleId": "932160",
          "file": "/path/to/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf",
          "lineNumber": "481",
          "data": "Matched Data: etc/passwd found within ARGS:x: /etc/passwd",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-shell",
            "platform-unix",
            "attack-rce",
            "paranoia-level/1",
            "OWASP_CRS",
            "capec/1000/152/248/88",
            "PCI/6.5.2"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      },
      {
        "message": "Inbound Anomaly Score Exceeded (Total Score: 10)",
        "details": {
          "match": "Matched \"Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `10' )",
          "reference": "",
          "ruleId": "949110",
          "file": "/path/to/rules/REQUEST-949-BLOCKING-EVALUATION.conf",
          "lineNumber": "81",
          "data": "",
          "severity": "2",
          "ver": "OWASP_CRS/3.3.5",
          "rev": "",
          "tags": [
            "application-multi",
            "language-multi",
            "platform-multi",
            "attack-generic"
          ],
          "maturity": "0",
          "accuracy": "0"
        }
      }
    ]
  }
}

Motivation

Detailed Logs in JSON format make it easy to store and analyze the logs in tools like elasticsearch or similiar. This increases the visibility of attacks, aids with filtering of false positives and greatly improves overall usefullness.

jptosso commented 1 year ago

Hey @amsnek , actually, Coraza supports a format called JSONLEGACY that is compatible with ModSecurity. https://github.com/corazawaf/coraza/blob/4f30afeca2b6cf48f8cd73acd80b072721172782/internal/auditlog/legacy.go#L9 Just replace the JSON format in SecAuditLogFormat with JSONLEGACY.

SecAuditLogFormat JSONLEGACY
amsnek commented 1 year ago 
SecAuditLogFormat JSONLEGACY

Hey @jptosso I tried with "SecAuditLogFormat JSONLEGACY" but the output did not change at all? 🤔 

{
  "transaction": {
    "time": "2023/08/07 10:02:15",
    "transaction_id": "c6c51035-6dc0-4edf-9e27-f9328397f04d",
    "remote_address": "127.0.0.1",
    "remote_port": 44113,
    "local_address": "127.0.0.1",
    "local_port": 80
  },
  "request": {
    "request_line": "  ",
    "headers": {
      "accept": "*/*",
      "host": "127.0.0.1",
      "user-agent": "curl/7.61.1"
    }
  },
  "response": {
    "status": 0,
    "protocol": "",
    "headers": {}
  },
  "audit_data": {
    "messages": null,
    "error_messages": null,
    "handler": "",
    "stopwatch": {
      "Combined": 0,
      "P1": 0,
      "P2": 0,
      "P3": 0,
      "P4": 0,
      "P5": 0,
      "Sr": 0,
      "Sw": 0,
      "L": 0,
      "Gc": 0
    },
    "response_body_dechunked": false,
    "producer": [
      "OWASP_CRS/4.0.0-rc1"
    ],
    "server": "",
    "engine_mode": "On"
  }
}
jptosso commented 1 year ago

Apparently jsonlegacy is using modsecurity 2 format. We would need another JSON formatter for libmodsecurity

amsnek commented 1 year ago

modsecurity2 was only "partial json". libmodsecurity3 implemented full json support. a "full" json formatter for libmodsecurity would be most usefull.

jcchavezs commented 1 year ago

Up to work on this @amsnek ?

amsnek commented 1 year ago

hey @jcchavezs as mentioned in the august 2023 meeting, sadly I do not have the required skills to implement this :(

M4tteoP commented 7 months ago

Just to keep tracking the evolution of this issue: https://github.com/corazawaf/coraza/pull/968 implemented some missing information