Closed kenoir closed 8 months ago
Follows https://github.com/wellcomecollection/wellcomecollection.org/pull/10533. Attempt to reduce IIIF API costs by reducing bad bot traffic.
This change initially only counts traffic matched in order to identify the scale of the issue and the impact of this change.
[!WARNING] This is applied and superseded by https://github.com/wellcomecollection/platform-infrastructure/pull/420, which has also been applied.
terraform plan
# module.iiif-waf-test.aws_wafv2_web_acl.acl will be created + resource "aws_wafv2_web_acl" "acl" { + arn = (known after apply) + capacity = (known after apply) + description = "Access control for the wellcomecollection.org CloudFront distributions" + id = (known after apply) + lock_token = (known after apply) + name = "iiif-cloudfront-acl-test" + scope = "CLOUDFRONT" + tags_all = (known after apply) + default_action { + allow { } } + rule { + name = "bot-control-rule-group" + priority = 4 + override_action { + count {} } + statement { + managed_rule_group_statement { + name = "AWSManagedRulesBotControlRuleSet" + vendor_name = "AWS" + managed_rule_group_configs { + aws_managed_rules_bot_control_rule_set { + inspection_level = "COMMON" } } + rule_action_override { + name = "CategoryAdvertising" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryArchiver" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryContentFetcher" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryEmailClient" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryHttpLibrary" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryLinkChecker" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryMiscellaneous" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryMonitoring" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryScrapingFramework" + action_to_use { + count { } } } + rule_action_override { + name = "CategorySearchEngine" + action_to_use { + count { } } } + rule_action_override { + name = "CategorySecurity" + action_to_use { + count { } } } + rule_action_override { + name = "CategorySocialMedia" + action_to_use { + count { } } } + rule_action_override { + name = "CategoryAI" + action_to_use { + count { } } } + rule_action_override { + name = "SignalAutomatedBrowser" + action_to_use { + count { } } } + rule_action_override { + name = "SignalKnownBotDataCenter" + action_to_use { + count { } } } + rule_action_override { + name = "SignalNonBrowserUserAgent" + action_to_use { + count { } } } } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-bot-control-test" + sampled_requests_enabled = true } } + rule { + name = "core-rule-group" + priority = 1 + override_action { + count {} } + statement { + managed_rule_group_statement { + name = "AWSManagedRulesCommonRuleSet" + vendor_name = "AWS" } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-core-test" + sampled_requests_enabled = true } } + rule { + name = "known-bad-inputs-rule-group" + priority = 3 + override_action { + count {} } + statement { + managed_rule_group_statement { + name = "AWSManagedRulesKnownBadInputsRuleSet" + vendor_name = "AWS" } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-known-bad-inputs-test" + sampled_requests_enabled = true } } + rule { + name = "managed-ip-blocking" + priority = 0 + override_action { + count {} } + statement { + managed_rule_group_statement { + name = "AWSManagedRulesAmazonIpReputationList" + vendor_name = "AWS" } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-ip-block-test" + sampled_requests_enabled = true } } + rule { + name = "sqli-rule-group" + priority = 2 + override_action { + count {} } + statement { + managed_rule_group_statement { + name = "AWSManagedRulesSQLiRuleSet" + vendor_name = "AWS" } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-sqli-test" + sampled_requests_enabled = true } } + visibility_config { + cloudwatch_metrics_enabled = true + metric_name = "iiif-cloudfront-acl-metric-test" + sampled_requests_enabled = true } } Plan: 1 to add, 4 to change, 0 to destroy.
What's changing and why?
Follows https://github.com/wellcomecollection/wellcomecollection.org/pull/10533. Attempt to reduce IIIF API costs by reducing bad bot traffic.
This change initially only counts traffic matched in order to identify the scale of the issue and the impact of this change.
terraform plan
diff