DataBiosphere / azul

Metadata indexer and query service used for AnVIL, HCA, LungMAP, and CGP
Apache License 2.0
7 stars 2 forks source link

Set audit record retention period #4043

Open theathorn opened 2 years ago

theathorn commented 2 years ago

From SSP AU-11 Audit Record Retention: The organization retains audit records for [FedRAMP Assignment: at least ninety (90) days] to provide support for after-the-fact investigations of security incidents and to meet regulatory and organizational information retention requirements.

For CloudTrail Logs, it is planned to have its audit retention record set to minimum one year.

melainalegaspi commented 2 years ago

@hannes-ucsc : "This is a cross-cutting concern. We will apply the required retention on any CloudWatch log groups we create in the future. Once we are closer to a full implementation of our logging and monitoring architecture, we will review all log groups again in a spike and fix those that need longer retention."

nolunwa-ucsc commented 1 year ago

Planned: review audit log retention across to the system and set it to 90 days on cloudwatch and other services where long retention might cause excessive cost (for example cloudtrail) and 1 year for others (for example S3 access log)

dsotirho-ucsc commented 1 year ago

CloudWatch logs retention is currently set to 180 days for most logs https://github.com/DataBiosphere/azul/blob/d50be3b52eb4f57603b1cb673697bcac35425935/src/azul/__init__.py#L257 and 30 days for ElasticSearch error logs https://github.com/DataBiosphere/azul/blob/d50be3b52eb4f57603b1cb673697bcac35425935/src/azul/__init__.py#L259