cisagov / ScubaGear

Automation to assess the state of your M365 tenant against CISA's baselines
https://www.cisa.gov/resources-tools/services/secure-cloud-business-applications-scuba-project
Creative Commons Zero v1.0 Universal
1.76k stars 226 forks source link

Improve functional test notifications #1401

Closed james-garriss closed 1 week ago

james-garriss commented 2 weeks ago

๐Ÿ—ฃ Description

This changes how notifications by email work when functional tests fail. Instead of emailing the TCOs once per test plan failure, it now only emails once per product failure. Additionally, it adds a link directly to the GitHub workflow job, making it easier for TCOs to review the failures.

๐Ÿ’ญ Motivation and context

The first goal is to reduce email fatigue. The second goal is to make it easier for TCOs to find the problem.

Closes: #1365

๐Ÿงช Testing

Created a test workflow to identify how to create the URL and to send email only if one or more previous jobs failed. Transferred this knowledge to the functional test workflow.

โœ… Pre-approval checklist

Almost. I fixed a long-standing error in omit_policies.yaml where it had the wrong EOL marker.

โœ… Pre-merge checklist

โœ… Post-merge checklist

james-garriss commented 1 week ago

@nanda-katikaneni