PHACDataHub / cpho-phase2

A data collection and retrieval application to automate and standardize the data intake for the yearly Health of Canadians report
5 stars 3 forks source link

Add cleanup policies for Artifact Registry #297

Closed vedantthapa closed 5 months ago

vedantthapa commented 6 months ago

Clean up policies can be configured for automatically deleting artifact versions that we no longer need or keeping artifacts that we want to store indefinitely.

This should be implemented as part of the artifact registry's IaD configuration.

Documentation: https://cloud.google.com/artifact-registry/docs/repositories/cleanup-policy

vedantthapa commented 6 months ago

I was wondering if something like - "Delete artifacts older than 30 days but keep 5 most recent versions" would work?

Any suggestions @AlexCLeduc / @Stephen-ONeil ?

Stephen-ONeil commented 6 months ago

Sounds good to me.

My only immediate concern was that we may lose (easy) access to vulnerability scan reports for deleted images, which we may need to hold on to for auditing. I dug through some docs and it seems vulnerability scan reports are stored in a project-level Grafeas database where general project metadata goes. Didn't find anything about the retention time of records there, or if deleting the images cascades to delete their metadata too (unlikely?), but I did find audit logs for each vulnerability record creation, associated to the scanned image's URI. Worst case, we have those retained for our standard audit log storage period so we can answer questions like "were you potentially vulnerable to CVE xxx during time period yyy," even if we've already deleted the image :+1:

vedantthapa commented 6 months ago

Great catch, thanks for looking into this :)