YannickRe / azuredevops-buildagents

Generate self-hosted build agents for Azure DevOps, just like Microsoft does.
MIT License
151 stars 90 forks source link

Feature request - support for managed image library #18

Closed Marcus-James-Adams closed 1 year ago

Marcus-James-Adams commented 2 years ago

Azure now has the concept of a manged image library that allows sharing of images and automatic distribution across regions

https://docs.microsoft.com/en-us/azure/virtual-machines/shared-image-galleries

It would be nice if their was an option to optionally specify that you want to store the image in a manged library and what regions you would like it to be stored in.

It can be done post packer image generation, additionally; packer has support for this built in.

YannickRe commented 2 years ago

Hi @Marcus-James-Adams,

Sounds interesting but I don't have any requirement for this in my own use case.

erik-de-bont commented 1 year ago

Hi @YannickRe , @Marcus-James-Adams ,

First thanks @YannickRe for all your work on these pipelines!

The advantage of the azure vm compute gallery is better (or easier) version management. You can configure your VMSS to use the latest version of the image from your gallery, so you can rollback or change images without touching your VMSS configuration. You can also deploy images within all subscriptions in your tenant (currently in preview), and there is a preview option to share your images with the community (all other azure users). The last option however doesn't allow the images to be used in a VMSS.

Switching the VMSS to the new gallery image it just a couple of clicks in the azure portal. You select the required image (for example ubuntu2204-agentpool-full) from your gallery and the latest version via the Change Image Reference option under the Operating System setting.

You can find my implemention in my fork

I have implemented a solution by replacing the updatevmss job with a new createvmimageversion job in the buildagent-generation-template.yml pipeline, which will generate a new version in your azure compute gallery. I could not get it to work with packer, but I was able to get it to work via an import of the managed image. So packer will build the image, it will be converted to a managed image and then imported into the gallery. The managed image will be removed afterwards.

You need to create a VM Definitions in your gallery called: ubuntu2204-agentpool-full windows2019-agentpool-full etc...

The version number it will create is based on the date (yyyy.mm.dd) . So it will be something like 2023.01.10. The name of the gallery and it's resource group need to be added DevOps Variable Group (see README.md).

If you have questions or comments let me know.

YannickRe commented 1 year ago

@erik-de-bont thanks for this! It helps me understand what the steps are to move this into a Gallery. I can use your work, and the requirement of #30 to move to Managed Image from the start (instead of converting afterwards), and create a combined version that would have the flexibility of with or without a Gallery.

This will take a while due to time constraints, but I love your input here and it will significantly help this project move to the next level.

erik-de-bont commented 1 year ago

Hi @YannickRe !

I have worked last couple of days on creating a combined version (VMSS or Gallery VM version). I have made the following updates.

Split the pipelines up in the following stages:

  1. Build the Agent Pool VM
  2. Convert the VM to a Managed VM
  3. Connect the Managed VM to a VMSS, or update the GalleryVM image with a new version (depending on your input (vmss or galleryvm), when starting the pipeline.

In Azure DevOps it looks something like this.

image

The code should be complete, but I have to do some more testing. You can find the source here

YannickRe commented 1 year ago

This has now been implemented through #33 by @erik-de-bont