search

aws-solutions / aws-waf-security-automations

This solution automatically deploys a single web access control list (web ACL) with a set of AWS WAF rules designed to filter common web-based attacks.
https://aws.amazon.com/solutions/aws-waf-security-automations
Apache License 2.0
835 stars 358 forks source link

allow cloudwatch logs destination for traffic logs, currently only supported target is s3 #260

Open sbe-arg opened 6 months ago

sbe-arg commented 6 months ago

would be great to have a way to setup traffic logs destination to be cloudwatch or kinesis as currently only s3 destination is supported, it is okay but is not always the intended source specially for quick testing waf acls is easier to query logs on cloudwatch than athena

aijunpeng commented 6 months ago

Thank you for the suggestion. Currently querying CloudWatch logs is not in the scope of the solution as the main purpose of the log analysis feature is to automate detection and blocking of malicious IPs via a built-in Lambda or Athena log parser against logs in S3, instead of providing a way to query logs. If I missed anything, feel free to provide details and how you want to customize the solution for your use case.

sbe-arg commented 6 months ago

@aijunpeng I'm referring to this setting AWS::WAFv2::WebACLLoggingConfiguration https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-wafv2-loggingconfiguration.html wich is a critical part of ACL setups

aijunpeng commented 6 months ago

What do you want to do with CW logs? Run custom queries against them?

sbe-arg commented 6 months ago

Debug acl blocks in an easier way than athena queries

aijunpeng commented 6 months ago

I would need more details about debugging acl blocks in an easier way. Currently in the WAF solution, Athena queries are already implemented and customers don't need to write their own queries.

sbe-arg commented 2 months ago

I forgot about this issue until I had to set this up again for another client. Also as of 4.0.3 there is no way to set the redacted fields for s3 logs, leading to a drift.

It needs to support more log sources not just s3, at least cloudwatch with a desired log-group expiration for compliance or passing the log-group name at setup time, if the log-group is created by the toolset, default X days expiration and overwrite with other INT value.

firehose is very flow specific can probably be left out image

My client uses cloudwatch insights to query quickly the waf logs.

aijunpeng commented 2 months ago

Thank you for providing more information. I understand you want to add CloudWatch log group as a log source. We can add your request to our backlog for evaluation. Meanwhile I'd like to clarify a couple of things:

  1. The solution is not a log query tool (If that is what you want, this tool might not be the right choice for you). Instead it is intended to use AMRs or built-in log parser to parse the logs, identify and block malicious IPs.
  2. Firehose is already supported with S3 as its destination.
sbe-arg commented 2 months ago

Yes cloudwatch as option for log source