Automated Tenant Configuration for Testing

[schrolla]

💡 Summary

Create a mechanism for configuring M365 tenant services to support rapid, automated functional testing of ScubaGear. Specifically, leverage the existing M365 Desired State Configuration (DSC) PowerShell modules and develop a set of DSC configurations associated with developer functional tests.

Motivation and context

Validating that ScubaGear accurately assesses M365 service configurations requires testing it against a wide variety of tenant configurations to ensure adequate coverage of different tenant settings. Manually reconfiguring tenant settings is time consuming for developers and precludes use of an automated continuous testing pipeline without a human in the loop. Automating tenant configuration will speed testing of new features and help ensure regression bugs are caught before being merged into the released ScubaGear product.

Implementation notes

The implementation for automated configuration will rely on the ability to use service principals to execute DSC configurations and including the Microsoft365DSC PowerShell module and its dependencies into the build.

Developers will create a DSC configuration, in the form of a PowerShell script, that defines a set of M365 resources along with a set of key/value pairs that define service settings. Developers could run each configuration manually to support initial testing of a functional test case. However, the focus is on creating a GitHub action that given a DSC configuration and tenant as input will run the desired configuration. This is meant to support a larger functional testing pipeline that requests each tenant configuration as part of its functional test set.

Related work and issues includes creation of the larger functional testing pipeline automation.

Acceptance criteria

• [ ] DSC coverage against SCuBA M365 baseline policy settings determined and documented
• [ ] GitHub Action successfully runs Microsoft365DSC Start-DSCConfiguration for a specified config and test tenant
• [ ] Basic set of all pass/all fail tenant DSC configurations for each ScubaGear assessed product added to repo functional testing folder
• [ ] DSC GitHub Action successfully integrated into larger functional testing pipeline

[schrolla]

New automated testing framework provides the same capabilities for the current project needs. This issue is now duplicative. Closing.