search

weather-gov / weather.gov

weather.gov 2.0
Other
331 stars 10 forks source link

automatically monitor s3 bucket sizes #2102

Open jamestranovich-noaa opened 5 days ago

jamestranovich-noaa commented 5 days ago

Description

Currently we have no way to automatically monitor the sizes of our AWS S3 buckets. Unfortunately, New Relic requires Cloudwatch logs to monitor the S3 bucket size, which we can't setup because we have 1. no direct access to Cloudwatch and 2. these Cloudwatch metrics are separately billed on the AWS side.

The cloud.gov support team filed a ticket for ingesting Cloudwatch metrics for bucket size into their new logging system (based on OpenSearch). In the meantime, if we need to check the bucket size, we can do something like aws s3 ls --summarize --human-readable --recursive

Acceptance criteria

colinmurphy01 commented 3 days ago

Based on the files we're ingesting from DSS Builder is this a budget risk and something we should prioritize sooner than later?

jamestranovich-noaa commented 13 hours ago

I think it is OK to check sizes once in a while, but we should look for a more permanent monitoring solution once we have something in place. We have a 5TB limit on buckets and I don't anticipate us reaching that limit anytime soon.