Revise WAF rules to block more malicious access

[jsjiang]

We may need to revise custom WAF rules to restrict access to the end points that are not covered in the initial version.

For example the following accesses from 92.119.179.86 should all be blocked:

• Time period: Sept 25, 8-11am
• Total access: 50,751
• Allow: 48%
• Block: 52%
• Allowed URIs:

• /doc/
• /account/
• /demo/
• /static/locale/en
• /static/info/
• /
• /admin/login/
• /login
• Index.asp, index.jsp, index.cgi, index.htm, etc.

Action: Revise custom WAF rules to

• restrict access to doc, account, demo endpoints
• Block access to the index pages

[jsjiang]

[jsjiang]

10/3-10/10:

• over 11K access on '/status' and the majority came from Duke University. It looks like they have a process that accesses the /status page once every minute. Should we ask why they do this?
• over 8K access to '/webhooks/sf-express/order/update-order-status' from 47.75.202.47. This kind of access should be blocked.
• Do we need to do anything with access to the '/oai' page?

[jsjiang]

Limit access to the account/pwreset page:

• There were over 700 access to this page during 10/12- 10/17
• Some are GETs and some are POSTs
• 65% were allowed, 35% were blocked.
• The majority came from a few IP addresses

[jsjiang]

Investigate if we should limit access to the /oai

• on 10/11 there were over 25K access from IP address 178.25.156.200 (Berlin, Germany)

[jsjiang]

2024-07-17:

• Modify rate-limit rule to rate limiting more url patterns
• The rate-limit rule blocks accesses when requests for a certain URL patten from the same IP address is over 100 in a 5 minutes window

- Action:
    Block: {}
  Name: rate-limit
  Priority: 13
  Statement:
    RateBasedStatement:
      AggregateKeyType: IP
      Limit: 100
      ScopeDownStatement:
        RegexMatchStatement:
          FieldToMatch:
            UriPath: {}
          RegexString: (^\/login|^\/logout|^\/learn|^\/home|^\/account|^\/contact|^\/dashboard|^\/demo|^\/doc|^\/download|^\/manage|^\/search|^\/status|^\/version|^\/s3_download|^\/admin|^\/monitor|^\/$)
          TextTransformations:
          - Priority: 0
            Type: NONE
  VisibilityConfig:
    CloudWatchMetricsEnabled: true
    MetricName: rate-limit
    SampledRequestsEnabled: true

Added URL patterns:

d-bash-4.2$ diff uc3-ezidui-stg-waf.rules_output.20240717.yaml uc3-ezidui-stg-waf.rules.input.20240717_test_v1.yaml
192c192
<           RegexString: (^\/login|^\/learn|^\/contact|^\/$)
---
>           RegexString: (^\/login|^\/logout|^\/learn|^\/home|^\/account|^\/contact|^\/dashboard|^\/demo|^\/doc|^\/doanload|^\/manage|^\/search|^\/status|^\/version|^\/s3_download|^\/admin|^\/monitor|^\/$)
-bash-4.2$ 

• Updated uc3-ezidui-stg-waf

-bash-4.2$ ~/bin/modify-waf-rules.py uc3-ezidui-stg-waf -i uc3-ezidui-stg-waf.rules.input.20240717_test_v1.yaml --execute Updates to webacl 'uc3-ezidui-stg-waf' successful.

[jsjiang]

Created dashboard ezid-waf-stg_dashboard in OpenSearch to monitor/testing the new WAF rules.

[jsjiang]

Tested using the Locust load test tool accessing the rate limited URLs:

Locust testing function:

    @task()
    def others(self):
        urls = [
        "login", "logout",
        "learn", "home", "account", "contact","dashboard", "demo", "doc", "download", "manage", "search", 
        "status", "version", "s3_download", "admin", "monitor",
        ""
    ]
        for url in urls:
            res =  self.client.get(f"{url}")
            res =  self.client.get(f"{url}")
            print(f"url: {url}, status_code: {res.status_code}")

Rate limit rule was triggered on those defined URLs:

[adambuttrick]

Fine to move into prod.

[jsjiang]

2024-07-23:

• exported current prd setting and saved asuc3-ezidui-prd-waf.rules.output.20240723_before.yaml
• copied stg testing file to uc3-ezidui-prd-waf.rules.input.20240723.yaml for prd use
• compared the changes

-bash-4.2$ ~/bin/modify-waf-rules.py uc3-ezidui-prd-waf -o uc3-ezidui-prd-waf.rules.output.20240723_before.yaml

cp uc3-ezidui-stg-waf.rules.input.20240717_test_v1.yaml uc3-ezidui-prd-waf.rules.input.20240723.yaml

-bash-4.2$ diff uc3-ezidui-prd-waf.rules.output.20240723_before.yaml uc3-ezidui-prd-waf.rules.input.20240723.yaml
192c192
<           RegexString: (^\/login|^\/learn|^\/contact|^\/$)
---
>           RegexString: (^\/login|^\/logout|^\/learn|^\/home|^\/account|^\/contact|^\/dashboard|^\/demo|^\/doc|^\/download|^\/manage|^\/search|^\/status|^\/version|^\/s3_download|^\/admin|^\/monitor|^\/$)

-bash-4.2$ ~/bin/modify-waf-rules.py uc3-ezidui-prd-waf -i uc3-ezidui-prd-waf.rules.input.20240723.yaml
  - Name: AWS-AWSManagedRulesAmazonIpReputationList
    OverrideAction:
      None: {}
    Priority: 0
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesAmazonIpReputationList
        RuleActionOverrides:
        - ActionToUse:
            Block: {}
          Name: AWSManagedIPDDoSList
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesAmazonIpReputationList
      SampledRequestsEnabled: true
  - Name: AWS-AWSManagedRulesCommonRuleSet
    OverrideAction:
      None: {}
    Priority: 1
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesCommonRuleSet
        RuleActionOverrides:
        - ActionToUse:
            Count: {}
          Name: NoUserAgent_HEADER
        - ActionToUse:
            Count: {}
          Name: UserAgent_BadBots_HEADER
        - ActionToUse:
            Count: {}
          Name: SizeRestrictions_QUERYSTRING
        - ActionToUse:
            Count: {}
          Name: SizeRestrictions_BODY
        - ActionToUse:
            Count: {}
          Name: CrossSiteScripting_BODY
        - ActionToUse:
            Count: {}
          Name: CrossSiteScripting_URIPATH
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesCommonRuleSet
      SampledRequestsEnabled: true
  - Name: AWS-AWSManagedRulesKnownBadInputsRuleSet
    OverrideAction:
      None: {}
    Priority: 2
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesKnownBadInputsRuleSet
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesKnownBadInputsRuleSet
      SampledRequestsEnabled: true
  - Name: AWS-AWSManagedRulesLinuxRuleSet
    OverrideAction:
      None: {}
    Priority: 3
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesLinuxRuleSet
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesLinuxRuleSet
      SampledRequestsEnabled: true
  - Name: AWS-AWSManagedRulesPHPRuleSet
    OverrideAction:
      None: {}
    Priority: 4
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesPHPRuleSet
        RuleActionOverrides:
        - ActionToUse:
            Count: {}
          Name: PHPHighRiskMethodsVariables_BODY
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesPHPRuleSet
      SampledRequestsEnabled: true
  - Name: AWS-AWSManagedRulesWordPressRuleSet
    OverrideAction:
      None: {}
    Priority: 5
    Statement:
      ManagedRuleGroupStatement:
        Name: AWSManagedRulesWordPressRuleSet
        VendorName: AWS
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: AWS-AWSManagedRulesWordPressRuleSet
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: wordpress
    Priority: 8
    Statement:
      RegexMatchStatement:
        FieldToMatch:
          UriPath: {}
        RegexString: (\/wp-content\/|\/wp-config\/|\/wordpress\/|\/wp-admin\/|\/wp-includes\/|\/wp-json\/)
        TextTransformations:
        - Priority: 0
          Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: wordpress
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: phpcgi
    Priority: 9
    Statement:
      RegexMatchStatement:
        FieldToMatch:
          UriPath: {}
        RegexString: (\/cgi-bin\/|\.php)
        TextTransformations:
        - Priority: 0
          Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: phpcgi
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: dotfile
    Priority: 10
    Statement:
      RegexMatchStatement:
        FieldToMatch:
          UriPath: {}
        RegexString: (boot\.ini|win\.ini|\.env)
        TextTransformations:
        - Priority: 0
          Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: dotfile
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: unix
    Priority: 11
    Statement:
      RegexMatchStatement:
        FieldToMatch:
          UriPath: {}
        RegexString: (\/etc\/passwd|%2Fpasswd)
        TextTransformations:
        - Priority: 0
          Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: unix
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: escape
    Priority: 12
    Statement:
      RegexMatchStatement:
        FieldToMatch:
          UriPath: {}
        RegexString: (\\\\\\\\|\/\/\/\/\/)
        TextTransformations:
        - Priority: 0
          Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: escape
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: rate-limit
    Priority: 13
    Statement:
      RateBasedStatement:
        AggregateKeyType: IP
        Limit: 100
        ScopeDownStatement:
          RegexMatchStatement:
            FieldToMatch:
              UriPath: {}
-           RegexString: (^\/login|^\/learn|^\/contact|^\/$)
+           RegexString: (^\/login|^\/logout|^\/learn|^\/home|^\/account|^\/contact|^\/dashboard|^\/demo|^\/doc|^\/download|^\/manage|^\/search|^\/status|^\/version|^\/s3_download|^\/admin|^\/monitor|^\/$)
            TextTransformations:
            - Priority: 0
              Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: rate-limit
      SampledRequestsEnabled: true
  - Action:
      Allow: {}
    Name: root-access-allow
    Priority: 14
    Statement:
      AndStatement:
        Statements:
        - RegexMatchStatement:
            FieldToMatch:
              UriPath: {}
            RegexString: ^\/$
            TextTransformations:
            - Priority: 0
              Type: NONE
        - RegexMatchStatement:
            FieldToMatch:
              QueryString: {}
            RegexString: (current_profile)
            TextTransformations:
            - Priority: 0
              Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: root-access-allow
      SampledRequestsEnabled: true
  - Action:
      Block: {}
    Name: root-access-block
    Priority: 15
    Statement:
      AndStatement:
        Statements:
        - RegexMatchStatement:
            FieldToMatch:
              UriPath: {}
            RegexString: ^\/$
            TextTransformations:
            - Priority: 0
              Type: NONE
        - RegexMatchStatement:
            FieldToMatch:
              QueryString: {}
            RegexString: (^\S+$)
            TextTransformations:
            - Priority: 0
              Type: NONE
    VisibilityConfig:
      CloudWatchMetricsEnabled: true
      MetricName: root-access-block
      SampledRequestsEnabled: true

[jsjiang]

Applied updated rules on prd (2024-07-23):

-bash-4.2$ ~/bin/modify-waf-rules.py uc3-ezidui-prd-waf -i uc3-ezidui-prd-waf.rules.input.20240723.yaml --execute
Updates to webacl 'uc3-ezidui-prd-waf' successful.