search

Azure / PSRule.Rules.Azure

Rules to validate Azure resources and infrastructure as code (IaC) using PSRule.
https://azure.github.io/PSRule.Rules.Azure/
MIT License
387 stars 84 forks source link

Create automated PR workflow to bump providers.json #1041

Closed ArmaanMcleod closed 2 years ago

ArmaanMcleod commented 2 years ago

Create workflow to update providers.json automatically, preferrably with an automated PR from github actions that runs daily using a cron job.

We could do a JSON diff between the current providers.json from the main branch and the REST API response data. For this diff to be reliable, we need to compare compressed JSON with sorted keys.

If we find a change, we can run the ExportProviders task in pipeline.build.ps1 in a new branch checked out from main and raise a PR.

ArmaanMcleod commented 2 years ago

I guess the other option is to run ExportProviders first, then see if any git changes were picked up. I just think that could be unreliable since if properties are reordered or whitespace is added, it will count as a change. Probably unlikely this will happen but it could bring up false positivies.

BernieWhite commented 2 years ago

@ArmaanMcleod It's worth monitoring. I'm not noticing to much of that as there is some limited sorting already.

ArmaanMcleod commented 2 years ago

@BernieWhite After doing some testing in my fork, it seems like the providers API changes quite often. Although the most recent run from BumpProviders didn't pick up any changes, hence no PR was raised, which is expected. I must have done these runs when they were making frequent changes to the API 😄 .

My merges in my fork to show this: https://github.com/ArmaanMcleod/PSRule.Rules.Azure/commits/main.

My workflow: https://github.com/ArmaanMcleod/PSRule.Rules.Azure/blob/main/.github/workflows/providers.yml.

I kept it simple by just running Invoke-Build ExportProviders and picking up changes. Given this task already sorts, didn't see the point of having this pipeline do that again for comparison.

I'm more concerned about how frequent this pipeline could run, and if I also need to setup a secret SPN as I've done with AZURE_CREDENTIALS in the main repo.

I have the cron exresspion set to 0 */6 * * *, which is every 6 hours, which might be too frequent for this. Was thinking maybe once a a week/month or something

ArmaanMcleod commented 2 years ago

Strange, the API response seems to keep removing/adding the same data to the providers: https://github.com/ArmaanMcleod/PSRule.Rules.Azure/pull/8. I'm pretty sure I've merged these changes twice already.

BernieWhite commented 2 years ago

@ArmaanMcleod Hmm interesting. This could be a result of A/B testing against different provider versions. I think it doesn't really matter hugely if we runs on a monthly basis. We are only shipping a stable minor version once a month and ideally providers.json shouldn't change for a patch version.

Yes we would not be able to call ARM without authorization so we'll need to setup a secret.