ModSecurity Variables Automatically Expire After 1 Minute Without ExpireVar Setting

takumi-ricoh commented 2 days ago

Describe the bug

Variables created using setvar in ModSecurity rules are being automatically deleted after 1 minute, even without explicitly setting expirevar. This occurs when trying to implement custom rules for login attempt tracking without using CRS.

Steps to reproduce

Use the official OWASP ModSecurity CRS Docker image (owasp/modsecurity-crs:4.7.0-nginx-202410090410)

Override /etc/nginx/templates/modsecurity.d/setup.conf.template with custom configuration:

Comment out CRS includes:

# Include /etc/modsecurity.d/owasp-crs/crs-setup.conf
# Include /etc/modsecurity.d/owasp-crs/rules/*.conf

Add custom rule file:

Include /opt/on_pre_llm/extra_lockout_rule.conf

Implement custom rule for tracking login attempts:

SecRule REQUEST_URI "@beginsWith /console/api/login" \
   "id:100098,phase:1,pass,\
   chain"
   SecRule &IP.failed_attempts "@eq 0" \
   "setvar:IP.failed_attempts=0,\
   setvar:IP.is_locked=0,\
   msg:'Initialization - Failed attempts: %{IP.failed_attempts}'"

Wait for 1 minute
Make another request to the login endpoint

Expected behaviour

The IP.failed_attempts and IP.is_locked variables should persist indefinitely until explicitly cleared
The initialization message should only appear on the first request when the variables don't exist
Actual behaviour
Variables are automatically deleted after 1 minute
The initialization message appears again after 1 minute, indicating that the variables have been reset
This behavior occurs with both IP and GLOBAL collection variables
No expirevar is explicitly set in the rules

Additional context

This issue prevents implementing proper login lockout functionality as the attempt counter keeps resetting
Confirmed that custom rules are working as the messages appear in the Docker logs
CORS is not active and not related to this issue

Your Environment

CRS version: Not using CRS (disabled)
Paranoia level setting: N/A (custom rules only)
ModSecurity version: As included in owasp/modsecurity-crs:4.7.0-nginx-202410090410
Web Server and version: nginx (version from Docker image)
Operating System and version: Ubuntu 24.04
Docker image: owasp/modsecurity-crs:4.7.0-nginx-202410090410

fzipi commented 1 day ago

@theseion Do you have a clue on why this might be happening?

fzipi commented 1 day ago

@takumi-ricoh Are you sure you are using the right syntax? 🤔 What I'm seeing in CRS is that we use :. E.g.


 SecRule &IP:failed_attempts "@eq 0" \
    "setvar:'IP.failed_attempts=0',\
    setvar:'IP.is_locked=0',\
    msg:'Initialization - Failed attempts: %{IP.failed_attempts}'"

takumi-ricoh commented 1 day ago

@fzipi Thank you for the advice. I tried it but it still didn't work.

I believe I'm in a situation where the application I want to use has no login lockout feature, and I have to implement it through WAF, so I'd like to solve this using ModSecurity somehow.

※ Given that there's documentation like this, implementing these (brute-force) measures using ModSecurity isn't strange, right? https://docs.mirantis.com/mcp/q4-18/mcp-security-best-practices/use-cases/brute-force-prevention/create-brute-force-rules.html

Current Code

When using the following code, if accessed within 1 minute, the count increases and in the logs, "Current number of failures" incrementally goes up like 1⇒2⇒..., but after 1 minute passes, it resets back to 1.

print IP.failed_attempts as Current number of failures:

SecRule REQUEST_URI "@beginsWith /console/api/login" \
  "id:100098,phase:1,pass,\
  chain"
SecRule &IP:failed_attempts "@eq 0" \
  "setvar:'IP.failed_attempts=0',\
  setvar:'IP.is_locked=0',\
  msg:'Initialization - Failed attempts: %{IP.failed_attempts}'"

SecRule REQUEST_URI "@beginsWith /console/api/login" \
  "id:100102,phase:5,pass,\
  chain"
SecRule RESPONSE_STATUS "@rx ^4\d{2}$" \
  "setvar:'IP.failed_attempts=+1',\
  log,\
  msg:'Login failure count - Current number of failures: %{IP.failed_attempts}'"

docker log

※ ■■■■■■■■■■■■ hoge ■■■■■■■■■■■■ is my extra comment

1st access
- 100098 & 100102
- Current number of failures: 1
2nd access (immediately after)
- 100102
- Current number of failures: 2
3rd access (more than 1 minute after 2nd access)
- 100098 & 100102
- Current number of failures: 1
4th access (more than 1 minute after 3rd access)
- 100098 & 100102
- Current number of failures: 1

/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/90-copy-modsecurity-config.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/91-update-resolver.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/92-update-real_ip.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/93-update-proxy-ssl-config.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/94-activate-plugins.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/95-activate-rules.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/11/01 01:49:58 [warn] 1#1: "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/conf/server.crt"
nginx: [warn] "ssl_stapling" ignored, issuer certificate not found for certificate "/etc/nginx/conf/server.crt"
2024/11/01 01:49:58 [notice] 1#1: ModSecurity-nginx v1.0.3 (rules loaded inline/local/remote: 0/12/0)
2024/11/01 01:49:58 [notice] 1#1: libmodsecurity3 version 3.0.13
■■■■■■■■■■■■ This is 1st access　⇒　Current number of failures: 1 ■■■■■■■■■■■■ 
10.228.107.168 - - [01/Nov/2024:01:50:02 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov  1 01:50:02 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":59596,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042580233.389326","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:50:02 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 2nd access (immediately after)　⇒　Current number of failures: 2 ■■■■■■■■■■■■ 
10.228.107.168 - - [01/Nov/2024:01:50:04 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov  1 01:50:04 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":59596,"host_ip":"172.19.0.10","host_port":443,"unique_id":"17304258047.838869","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:50:04 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Login failure count - Current number of failures: 2","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 3rd access (more than 1 minute after 2nd access) 　⇒　Current number of failures: 1 ■■■■■■■■■■■■ 
10.228.107.168 - - [01/Nov/2024:01:51:14 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov  1 01:51:14 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":52208,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042587458.408275","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:51:14 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}
■■■■■■■■■■■■ 4th access (more than 1 minute after 3rd access)  　⇒　Current number of failures: 1 ■■■■■■■■■■■■ 
10.228.107.168 - - [01/Nov/2024:01:52:36 +0000] "POST /console/api/login HTTP/1.1" 400 84 "https://10.59.9.153/signin" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
{"transaction":{"client_ip":"10.228.107.168","time_stamp":"Fri Nov  1 01:52:36 2024","server_id":"3b6ba025dbb95abfb423609b369e4c857c659928","client_port":38452,"host_ip":"172.19.0.10","host_port":443,"unique_id":"173042595696.346467","request":{"method":"POST","http_version":1.1,"uri":"/console/api/login","headers":{"sec-ch-ua-mobile":"?0","Origin":"https://10.59.9.153","content-type":"application/json","Accept":"*/*","sec-ch-ua":"\"Chromium\";v=\"130\", \"Google Chrome\";v=\"130\", \"Not?A_Brand\";v=\"99\"","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36","Sec-Fetch-Site":"same-origin","sec-ch-ua-platform":"\"Windows\"","Referer":"https://10.59.9.153/signin","Content-Length":"64","Connection":"keep-alive","Sec-Fetch-Mode":"cors","authorization":"Bearer","Host":"10.59.9.153","Sec-Fetch-Dest":"empty","Accept-Encoding":"gzip, deflate, br, zstd","Cookie":"token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkNWFlYjc0LTQ1YTYtNDYzYi1iNzgwLTYyYmQwZTU4MzQxZiJ9.WIDBt2Va8W7bOo-v-thAxgZzGYJTSF9sA3U5V_OHS8g; zbx_session=eyJzZXNzaW9uaWQiOiJiMTNjZDI0NTc3MTllMTA1ZTM2NmQxMzFlNTY2YTczNiIsInNlcnZlckNoZWNrUmVzdWx0Ijp0cnVlLCJzZXJ2ZXJDaGVja1RpbWUiOjE3MzAzNjIyMzcsInNpZ24iOiIyNmI4MDYyYWIzZTQ5OTdhNTRlMDFkODE1MWQxNDNlODIwNzE3Mjc0OTE4MTE5ZWYyMGE5ZGQ5YTZiMmE3Njg3In0%3D; locale=ja-JP","Accept-Language":"ja,en-US;q=0.9,en;q=0.8"}},"response":{"body":"","http_code":400,"headers":{"Server":"nginx","Date":"Fri, 01 Nov 2024 01:52:36 GMT","Content-Length":"84","Content-Type":"application/json","X-Version":"0.7.3","Connection":"keep-alive","Set-Cookie":"remember_token=; Expires=Thu, 01 Jan 1970 00:00:00 GMT; Path=/","X-Env":"PRODUCTION"}},"producer":{"modsecurity":"ModSecurity v3.0.13 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":[]},"messages":[{"message":"Initialization - Failed attempts: 0","details":{"match":"Matched \"Operator `Eq' with parameter `0' against variable `IP:failed_attempts' (Value: `0' )","reference":"o0,18v5,18","ruleId":"100098","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"7","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}},{"message":"Login failure count - Current number of failures: 1","details":{"match":"Matched \"Operator `Rx' with parameter `^4\\d{2}$' against variable `RESPONSE_STATUS' (Value: `400' )","reference":"o0,18v5,18o0,3v1049,3","ruleId":"100102","file":"/opt/on_pre_llm/extra_lockout_rule.conf","lineNumber":"61","data":"","severity":"0","ver":"","rev":"","tags":[],"maturity":"0","accuracy":"0"}}]}}

coreruleset / modsecurity-crs-docker