Determine appropriate retention for network logs

[maxneuvians]

We currently have five types of network logs:

• Cloudfront logs
• WAF logs
• VPC flow logs
• ALB access logs
• Application (container) logs

Each should be tuned to operation requirements considering privacy and legal implications.

[obrien-j]

As a government service, we also have an obligation to protect against attacks on our system, particularly given the profile of this service. Network logs (including IP addresses) are a key tool in protecting against such attacks. While some threats can be identified quickly, some attacks aren't discovered right away, requiring analysis of logs over a longer period to recognize patterns of suspicious behaviour.

A retention period of 3 months provides responders with the ability to identify trends over time and to properly investigate and action any security breaches. Less than 3 months is not expected to reliably provide sufficient data to perform these actions.

That said, this process will be closely monitored. As we better understand how the system behaves in real-life scenarios, we may determine that the system can be defended with a shorter retention period. We would consider changes accordingly.

We recognize the increased sensitivity tied to collecting IP addresses, and appreciate that it's a fine line to walk. We restrict access to our logs—only authorized staff can access them for specific purposes, and their access is itself logged. Further, the IP addresses aren't connected to the actual data uploaded into the system—it's only associated with network logs.

We're aware that there are a spectrum of potential approaches we could take on this issue, and will continue to evaluate ours as we improve the system.