search

Azure / Azure-Verified-Modules

Azure Verified Modules (AVM) is an initiative to consolidate and set the standards for what a good Infrastructure-as-Code module looks like. Modules will then align to these standards, across languages (Bicep, Terraform etc.) and will then be classified as AVMs and available from their respective language specific registries.
https://aka.ms/AVM
MIT License
318 stars 65 forks source link

[Module Proposal]: `avm-res-devopsinfrastructure-pool` #1265

Closed jaredfholgate closed 9 hours ago

jaredfholgate commented 1 month ago

Check for previous/existing GitHub issues/module proposals

Check this module doesn't already exist in the module indexes

Bicep or Terraform?

Terraform

Module Classification?

Resource Module

Module Name

avm-res-devopsinfrastructure-pool

Module Details

This is a resource module for Managed DevOps Pools. It is currently in public preview, but will be a great solution to have available ASAP.

API ref: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/devopsinfrastructure/resource-manager/Microsoft.DevOpsInfrastructure/preview/2024-04-04-preview/devopsinfrastructure.json

ARM example: https://learn.microsoft.com/en-us/azure/devops/managed-devops-pools/quickstart-arm-template?view=azure-devops

This will be leveraged in the avm-ptn-cicd-agents-and-runners pattern module.

Do you want to be the owner of this module?

Yes

Module Owner's GitHub Username (handle)

jaredfholgate

(Optional) Secondary Module Owner's GitHub Username (handle)

No response

jaredfholgate commented 1 month ago

Cross referencing the issue for the Bicep version: https://github.com/Azure/Azure-Verified-Modules/issues/1217

johnlokerse commented 1 month ago

Mentioning @haflidif, who is already working on a Terraform MDP solution.

haflidif commented 1 month ago

@jaredfholgate I'm happy to collaborate with you on this, as I've already started 💯

in addition, I think the name avm-res-devopsinfrastructure-managed-devops-pools or avm-res-devopsinfrastructure-managed-pools might be more convenient. what do you think?

jaredfholgate commented 1 month ago

@jaredfholgate happy to collaborate with you on this 💯

Hello again @haflidif. :) Sure, I have a meeting about it on Monday and will get in touch. If you have a repo for it already, please feel free to share.

haflidif commented 1 month ago

@jaredfholgate happy to collaborate with you on this 💯

Hello again @haflidif. :) Sure, I have a meeting about it on Monday and will get in touch. If you have a repo for it already, please feel free to share.

I'll just wrap up a few commits and do a little cleanup in the repo, I'll share as soon as possible

haflidif commented 1 month ago

@jaredfholgate what do you think about the naming of the module, when I spoke to the PMs when I gave my repo a name, they liked the avm-res-devopsinfrastructure-managed-devops-pools - but maybe avm-res-devopsinfrastructure-managed-pools is more convenient ?

haflidif commented 1 month ago

@jaredfholgate what do you think about the naming of the module, when I spoke to the PMs when I gave my repo a name, they liked the avm-res-devopsinfrastructure-managed-devops-pools - but maybe avm-res-devopsinfrastructure-managed-pools is more convenient ?

nvm saw a request to have the Bicep module named the same way you did, so I will change the name of my repo to reflect the naming in the proposal 😎

jaredfholgate commented 1 month ago

@jaredfholgate what do you think about the naming of the module, when I spoke to the PMs when I gave my repo a name, they liked the avm-res-devopsinfrastructure-managed-devops-pools - but maybe avm-res-devopsinfrastructure-managed-pools is more convenient ?

nvm saw a request to have the Bicep module named the same way you did, so I will change the name of my repo to reflect the naming in the proposal 😎

Yeah, we are supposed to follow the api naming for these modules and I think this is right, but the PM can confirm and I can rename the repo if needed.

haflidif commented 1 month ago

@jaredfholgate Still a work in progress, but here is what I have so far: https://github.com/haflidif/terraform-azurerm-avm-res-devopsinfrastructure-pools

I've done some unit-tests on it and it seems to go through and deploy what I want - I will include the unit-test later today

Need to work on more documentation, and haven't yet run the avm.bat to create the documentation etc, but as I said, work in progress

johnlokerse commented 1 month ago

Hey @haflidif @jaredfholgate! Since we have to run the AVM Bicep/Terraform modules through automated tests I wonder what the best way is to have a connection to Azure DevOps. At some point in the deployment the user (in this case a SPN) that orchestrates the deployment needs to deploy an agent pool in an existing Azure DevOps organisation.

What are your views on this? Or is the approach different for Terraform?

cc @AlexanderSehr

jaredfholgate commented 1 month ago

Hey @haflidif @jaredfholgate! Since we have to run the AVM Bicep/Terraform modules through automated tests I wonder what the best way is to have a connection to Azure DevOps. At some point in the deployment the user (in this case a SPN) that orchestrates the deployment needs to deploy an agent pool in an existing Azure DevOps organisation.

What are your views on this? Or is the approach different for Terraform?

cc @AlexanderSehr

Terraform is already sorted as I had to implement a solution last week for Azure DevOps and GitHub. Basically, if you supply any GitHub variables or secrets that start with TF_VAR_, it will add them to the container environment, so you can declare variables in the examples (e2e tests) and supply them by this means.

For local testing use a tfvars file. For e2e tests we have orgs that can be used and the owner can set the variables in the repo or environment.

For Bicep, I am unsure but could potentially follow a similar approach.

jaredfholgate commented 1 month ago

Hey @haflidif @jaredfholgate! Since we have to run the AVM Bicep/Terraform modules through automated tests I wonder what the best way is to have a connection to Azure DevOps. At some point in the deployment the user (in this case a SPN) that orchestrates the deployment needs to deploy an agent pool in an existing Azure DevOps organisation. What are your views on this? Or is the approach different for Terraform? cc @AlexanderSehr

Terraform is already sorted as I had to implement a solution last week for Azure DevOps and GitHub. Basically, if you supply any GitHub variables or secrets that start with TF_VAR_, it will add them to the container environment, so you can declare variables in the examples (e2e tests) and supply them by this means.

For local testing use a tfvars file. For e2e tests we have orgs that can be used and the owner can set the variables in the repo or environment.

For Bicep, I am unsure but could potentially follow a similar approach.

Having said that, I now see that it can't use a PAT and requires a principal of some kind to register the pool in Azure DevOps. I'll have to figure that part out... Can possible grant the MSI perms during the test run I think.

matebarabas commented 1 month ago

@jaredfholgate, I've changed the module's name in this proposal to its singular form (pools --> pool), since as per the naming convention for resource modules, the RT segment of the name must be in singular form.

@prjelesi, please make sure the singular form is captured when this proposal is triaged. Thanks!

matebarabas commented 1 month ago

Official public preview announcement from the PG: https://aka.ms/hellomdp

jaredfholgate commented 1 month ago

@matebarabas and @prjelesi This module has now been released here if you want to close this issue and update the index? It is here: https://registry.terraform.io/modules/Azure/avm-res-devopsinfrastructure-pool

AlexanderSehr commented 1 month ago

Hey @haflidif @jaredfholgate! Since we have to run the AVM Bicep/Terraform modules through automated tests I wonder what the best way is to have a connection to Azure DevOps. At some point in the deployment the user (in this case a SPN) that orchestrates the deployment needs to deploy an agent pool in an existing Azure DevOps organisation. What are your views on this? Or is the approach different for Terraform? cc @AlexanderSehr

Terraform is already sorted as I had to implement a solution last week for Azure DevOps and GitHub. Basically, if you supply any GitHub variables or secrets that start with TF_VAR_, it will add them to the container environment, so you can declare variables in the examples (e2e tests) and supply them by this means.

For local testing use a tfvars file. For e2e tests we have orgs that can be used and the owner can set the variables in the repo or environment.

For Bicep, I am unsure but could potentially follow a similar approach.

In Bicep we'll need to do something similar. As already discussed in a different location, we'll have to enable the BRM (ABM-Bicep) CI to dynamically pull secerets and pass them into templates. It should not be too complicated to implement, but needs to be aligned with the other Bicep maintainers. I hope we can get to discuss it tomorrow. It's useful for quite a few modules that need specific secrets / tenant-specific values.

matebarabas commented 1 month ago

@matebarabas and @prjelesi This module has now been released here if you want to close this issue and update the index? It is here: https://registry.terraform.io/modules/Azure/avm-res-devopsinfrastructure-pool

As @PmeshramPM is managing TF module proposals at this stage, I'd like to ask him to follow up on the conclusion of this development. Thanks!

prjelesi commented 1 month ago

@jaredfholgate i supose that you agree to be owner of this module and to follow all roles and responsibilities as module owner?

We just want to confirm you agree to the below pages that define what module ownership means:

Any questions or clarifications needed, let us know!

If you agree, please just reply to this issue with the exact sentence below (as this helps with our automation 👍):

"I CONFIRM I WISH TO OWN THIS AVM MODULE AND UNDERSTAND THE REQUIREMENTS AND DEFINITION OF A MODULE OWNER"

Thanks,

The AVM Core Team

jaredfholgate commented 1 month ago

I CONFIRM I WISH TO OWN THIS AVM MODULE AND UNDERSTAND THE REQUIREMENTS AND DEFINITION OF A MODULE OWNER

microsoft-github-policy-service[bot] commented 1 week ago

[!IMPORTANT] @jaredfholgate, this issue has not had any activity in the last 3 weeks. Please feel free to reach out to the AVM core team should you have any questions or need any help with the development of this module.

[!TIP] To silence this notification, provide an update every 3 weeks on the Module Proposal issue, or add the "Status: Long Term :hourglass_flowing_sand:" label.

prjelesi commented 9 hours ago

Hi @jaredfholgate

Thanks for confirming that you wish to own this AVM module and understand the related requirements and responsibilities!

Before starting development, please ensure ALL the following requirements are met.

Please use the following values explicitly as provided in the module index page:

Check if this module exists in the other IaC language. If so, collaborate with the other owner for consistency. 👍

You can now start the development of this module! ✅ Happy coding! 🎉

Please respond to this comment and request a review from the AVM core team once your module is ready to be published! Please include a link pointing to your PR, once available. 🙏

Any further questions or clarifications needed, let us know!

Thanks,

The AVM Core Team