Spike: Low Priority Pager duty alerts - review

[SimonPPledger]

User Story

As a modernisation platform engineer I want to look at our low priority pagerduty alerts So that we are only receiving alerts that require a response. However this might not be possible. This is to confirm what we want to do with these alerts rather than fix them (for now)

Value / Purpose

We regularly see the following alarms in our #modernisation-platform-low-priority-alarms Slack channel:

Sum unauthorised-api-calls GreaterThanOrEqualToThreshold 1.0

Sum sign-in-failures GreaterThanOrEqualToThreshold 1.0

Sum iam-policy-changes GreaterThanOrEqualToThreshold 1.0

This ticket will involve taking a look at the alerts we receive in the #modernisation-platform-low-priority-alarms channel and assessing if we need to be alerted for them.

Ideally we should only receive alerts for incidents that require our attention.

Useful Contacts

No response

Additional Information

We should ensure that we don't compromise our adherence to AWS Security Hub Security Standards.

Proposal / Unknowns

No response

Definition of Done

• [ ] Alarms investigated and triggers identified and what the thresholds are
• [ ] We agree whether these are necessary
• [ ] If not necessary, look at options to stop alerting and resolve cause if possible
• [ ] If necessary raise new tickets to resolve the cause of the alert

[dms1981]

Are these linked to Security Hub Control Standards? If we modify these alerts, will we affect our Security Hub compliance metric? Could we just not push these alerts through to Slack?

[ewastempel]

For Sum iam-policy-changes GreaterThanOrEqualToThreshold 1.0 make it specific to IAM policies that are used in our pipelines/SSO roles (all the roles that we own).

[ewastempel]

For Sum unauthorised-api-calls GreaterThanOrEqualToThreshold 1.0 and Sum sign-in-failures GreaterThanOrEqualToThreshold 1.0 adjust the threshold to notify when it happens on mass.

[SimonPPledger]

believe this is replaced by https://github.com/ministryofjustice/modernisation-platform/issues/1535 - but want to keep it open until we have confirmation