Open xopham opened 7 months ago
it is possible to use semgreps own validation: https://semgrep.dev/docs/writing-rules/testing-rules#validating-rules
semgrep scan --metrics=off --validate --config /app/rules --json -o broken_rules.json
Broken rules would result in the following broken_rules.json
:
{
"errors": [
{
"code": 4,
"level": "error",
"long_msg": "One of these properties is missing: 'languages'",
"short_msg": "Invalid rule schema",
"spans": [
{
"end": {
"col": 1,
"line": 27,
"offset": -1
},
"file": "rules/deny-default-namespace.yaml",
"source_hash": "c16ac57d9db7bb7c762e3775cf1982c20eb2161542acf11b32b04edc26730dea",
"start": {
"col": 3,
"line": 2,
"offset": -1
}
}
],
"type": "InvalidRuleSchemaError"
},
{
"code": 2,
"level": "error",
"message": "Semgrep match found at line ./rules/deny-default-namespace.yaml:2:\n Please include a 'languages' field for your rule $RULEID!",
"path": "./rules/deny-default-namespace.yaml",
"type": "Semgrep match found"
},
{
"code": 2,
"level": "error",
"message": "Rule parse error in rule restrict-image-registry:\n Missing required field regex",
"rule_id": "restrict-image-registry",
"type": "Rule parse error"
}
],
"paths": {
"scanned": []
},
"results": [],
"skipped_rules": [],
"version": "1.68.0"
}
There is no definite way to determine the path to a broken rule when scanning multiple rules.
In summary, some considerations are necessary:
/app/rules/
and validating afterwards might create race conditions where some deployments may fail. A 3-step process might be required: write to files in /app/dummy-rules
, run validation, copy to /app/rules
(incl. removal of old rules)p/semgrep-rule-lints
(running those directly like semgrep scan --config p/semgrep-rule-lints rules
does NOT yield the same result). Validation fails if semgrep API is unavailable. Air-gaped systems cannot perform validation which might create need for making validation configurable.
semgr8s loads rules from remote and configmaps. a single broken rule could break the policy engine as a whole. rules are also frequently updated. rules must be scanned against a schema to ensure broken rules cannot enter the system.
also important: document this schema, specifically 1 rule in 1 configmap, multiple rules in multiple configmaps, multiple rules in 1 configmap