Incomplete parser model and sample application for parsing Core Rule Set written in the ModSecurity DSL SecRule language. It uses the python library textX for parsing.
Install dependencies
Dependencies can be installed system-wide, or just for your user (using --user
).
System-wide:
sudo pip install secrules-parsing
User:
pip install --user secrules-parsing
Execute secrules-parser
specifying the location of the files you want to scan using the -f/--files argument. This takes wildcards or individual files.
$ secrules-parser -c -f /owasp-crs/rules/*.conf
Add flags to accomplish needed tasks:
-h, --help:
Description: show the help message and exit
Example: $ secrules-parser -h
-r, --regex:
Description: Extract regular expressions from rules file
Example:
$ secrules-parser --regex -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
{"/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf": [{"920100": ["^(?i:(?:[a-z]{3,10}\\s+(?:\\w{3,7}?://[\\w\\-\\./]*(?::\\d+)?)?/[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?|connect (?:\\d{1,3}\\.){3}\\d{1,3}\\.?(?::\\d+)?|options \\*)\\s+[\\w\\./]+|get /[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?)$"]}, {"920120": ["(?<!&(?:[aAoOuUyY]uml)|&(?:[aAeEiIoOuU]circ)|&(?:[eEiIoOuUyY]acute)|&(?:[aAeEiIoOuU]grave)|&(?:[cC]cedil)|&(?:[aAnNoO]tilde)|&(?:amp)|&(?:apos));|['\\\"=]"]}, {"920160": ["^\\d+$"]}, {"920170": ["^(?:GET|HEAD)$"]}, {"920171": ["^(?:GET|HEAD)$"]}, {"920180": ["^POST$"]}, {"920190": ["(\\d+)\\-(\\d+)\\,"]}, {"920210": ["\\b(?:keep-alive|close),\\s?(?:keep-alive|close)\\b"]}, {"920220": ["\\%(?:(?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920240": ["^(?:application\\/x-www-form-urlencoded|text\\/xml)(?:;(?:\\s?charset\\s?=\\s?[\\w\\d\\-]{1,18})?)??$"]}, {"920260": ["\\%u[fF]{2}[0-9a-fA-F]{2}"]}, {"920290": ["^$"]}, {"920310": ["^$"]}, {"920311": ["^$"]}, {"920330": ["^$"]}, {"920340": ["^0$"]}, {"920350": ["^[\\d.:]+$"]}, {"920420": ["^(?:GET|HEAD|PROPFIND|OPTIONS)$"]}, {"920440": ["\\.(.*)$"]}, {"920450": ["^.*$"]}, {"920200": ["^bytes=(?:(?:\\d+)?\\-(?:\\d+)?\\s*,?\\s*){6}"]}, {"920230": ["\\%((?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920121": ["['\\\";=]"]}, {"920460": ["(?<!\\Q\\\\\\E)\\Q\\\\\\E[cdeghijklmpqwxyz123456789]"]}]}
-c, --correctness:
Description: Check the validity of the syntax
Example:
$ secrules-parser -c -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Syntax OK: ../../../rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
-v, --verbose
Description: Print verbose messages
Example:
$ secrules-parser -c -v -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
...
-o FILE, --output FILE
Description: Output results to file
Example:
$ secrules-parser -c -o out.json -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
--output-type github | plain
Description: Desired output format. Useful if running from Github Actions and you want annotated output
Example:
$ secrules-parser -c --output-type github -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
Takes a list of file path's and returns models
import glob
import os
from secrules_parsing import parser
# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
import glob
import os
from secrules_parsing import parser
# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
parser.get_correctness(files, models)
If you want to modify this module, follow these steps:
git clone git@github.com:coreruleset/secrules_parsing.git
poetry install
first!tests
subdirectoryTo visualize the syntax tree, use:
textx visualize secrules.tx
dot -Tpng -O secrules.tx.dot
Then review the generated PNG modsec.tx.dot.png!
Please file an issue if you find a bug or you want some feature added.