Store stakeholder CyHy reports in AWS S3

[mcdonnnj]

💡 Summary

We should store any CyHy stakeholder reports in AWS S3 instead of in an EBS volume attached to the reporter instance.

Motivation and context

Storing the reports in S3 would allow the following advantages:

• It would enable us to use an S3 lifecycle policy to automatically transition to a different storage class or even delete an object.
• It would allow us to provide access to reports to CyHy team members in a straightforward, on-demand way.
  • Adding on to this we would no longer need to copy a CyHy team email address to provide the reports which would reduce the load on that mailbox.
• S3 is easier to modify on-demand than EBS if we need to increase storage capacity etc.

Implementation notes

We will need to create a new bucket to store reports (and this could probably be extended to store any reports in the CyHy environment). The reporter instance profile will need to be updated to allow PutObject permissions to this new bucket. I believe we would like a transition period where reports are copied from the EBS volume attached to the reporter instance to the new S3 bucket. At some future point the cisagov/cyhy-reports logic should be changed to upload directly to the bucket.

Acceptance criteria

How do we know when this work is done?

• [ ] New S3 bucket to house reports is created.
• [ ] The reporter instance is given appropriate permissions to allow reports to be uploaded to the new S3 bucket.
• [ ] A script or modifications to cisagov/cyhy-reports should be made for the transition period where data will live on the existing EBS volume but will also be copied to S3.

[mcdonnnj]

We might consider just mounting the S3 bucket to the local filesystem as a final state using mountpoint.