search

NASA-PDS / registry-sweepers

Scripts that run regularly on the registry database, to clean and consolidate information
Apache License 2.0
0 stars 1 forks source link

Registry-Sweeper ECS Enchancements (Pre Multi-Tenancy) #59

Closed sjoshi-jpl closed 9 months ago

sjoshi-jpl commented 10 months ago

💡 Description

All changes will be in JPL AWS.

SA team has created a registry-sweeper prod cluster for ECS in JPL AWS and following enhancements are required :

  1. Move all task definitions to point to new registry-sweeper prod cluster.
  2. Update all cloudwatch alarms and metrics to pull data from new registry-sweeper cluster. Also update ECS CloudWatch dashboard.
  3. Update EventBridge schedules to point to the new registry-sweeper prod cluste
  4. Turn on container insights for new cluster and disabled / delete old cluster (once approved by @tloubrieu-jpl or @jordanpadams
  5. Ensure the new CloudWatch Alarms / SNS notifications refer to registry-sweeper terminology rather than provenanc
  6. Update documentation for ECS in Private Wiki with new task-definition and cluster names.
  7. Do we need to create new alarms for monitoring registry-sweeper script failures like we had one for provenance?
sjoshi-jpl commented 10 months ago

For point #7 above we can leverage the same lambda based solution that we have for registry-api (#366). Instead of having 10 different alarms being triggered (for one task per node) we can have a single lambda monitor the logs and send an SNS notification for dev teams to investigate.

tloubrieu-jpl commented 10 months ago

lambda profiles have been created.