search

actions / runner-images

GitHub Actions runner images
MIT License
10k stars 3.02k forks source link

Ubuntu 24.04 image not compatible with Azure DevOps self-hosted VMSS #10122

Closed asbiin closed 2 months ago

asbiin commented 3 months ago

Description

When deploying the Ubuntu 24.04 image within an Azure virtual machine scale set for a VMSS pool of Azure DevOps, the Microsoft.Azure.DevOps.Pipelines.Agent extension is added, which is using python to run scripts.

Ubuntu 24.04 default python is now 3.12, which does not have the imp module anymore. When a VM starts and tries to run the extension, it fails with this error:

Traceback (most recent call last):
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.23.0.1/./AzureRM.py", line 9, in <module>
    import Utils.HandlerUtil as Util
  File "/var/lib/waagent/Microsoft.VisualStudio.Services.TeamServicesAgentLinux-1.23.0.1/Utils/HandlerUtil.py", line 58, in <module>
    import imp
ModuleNotFoundError: No module named 'imp'

The issue is more on vsts-azurevm-extension: https://github.com/microsoft/vsts-azurevm-extension/issues/233 but I add it here as a reference

Platforms affected

Runner images affected

Image version and build link

N/A

Is it regression?

no

Expected behavior

The VM can register to Azure DevOps host

Actual behavior

The extension starts and register the VM under the Azure DevOps pool

Repro steps

Create a self-hosted Azure DevOps pool, of type Azure virtual machine scale set, connected to your VMSS. Start a pipeline job using this pool.

vidyasagarnimmagaddi commented 3 months ago

Hi @asbiin , We will be analysing the issue and provide the solution. thanks

chirangaalwis commented 2 months ago

Hi folks,

Any update on this? We have faced the same issue when using a custom built VM image based on Ubuntu 24.04 LTS for a DevOps self hosted agent VMSS and are currently, unable to proceed.

We assume that downgrading the Python version is not the best approach as the currently installed Python version is installed by default within an Ubuntu distribution and could cause issues as it may already be in use.

tkasparek commented 2 months ago

The issue has been fixed and the fix has been rolled out everywhere, more in https://github.com/microsoft/vsts-azurevm-extension/issues/233

markDmullins commented 2 months ago

The issue has been fixed and the fix has been rolled out everywhere, more in microsoft/vsts-azurevm-extension#233

Unfortunately we are still experiencing this issue. When you create a new Azure virtual machine scale set agent pool in Azure DevOps it adds the extension with version 1.23 of TeamServicesAgentLinux that still imports imp. rather than version 1.25.2 from the fix you referenced.

image

This is from a brand new Azure VMSS and Agent Pool created today.

Is there some lead time before this propagates through to Azure DevOps or is an additional change needed?

DevOpsAzurance commented 2 months ago

Any updates?

We have a cron tab that rebuilds our images and we have had to roll back the latest version to avoid errors.

vidyasagarnimmagaddi commented 2 months ago

Hi @tkasparek , the issue is fixed in (https://github.com/microsoft/vsts-azurevm-extension/issues/233). Hence we are closing it . thank you

gavintoddchurch commented 2 months ago

Was there ever a solution to this? My extension is still the 1.23 version and I can't seem to upgrade to the version with the fix.

DevOpsAzurance commented 2 months ago

Still having this issue, building both off the latest release and main. This is my VMSS Extension version per 'az vmss extension list':

{ "autoUpgradeMinorVersion": false, "enableAutomaticUpgrade": null, "forceUpdateTag": null, "id": null, "name": "Microsoft.Azure.DevOps.Pipelines.Agent", "protectedSettings": null, "protectedSettingsFromKeyVault": null, "provisionAfterExtensions": null, "provisioningState": null, "publisher": "Microsoft.VisualStudio.Services", "settings": { "agentDownloadUrl": "https://vstsagentpackage.azureedge.net/agent/3.241.0/vsts-agent-linux-x64-3.241.0.tar.gz", "agentFolder": "/agent", "enableScriptDownloadUrl": "https://vstsagenttools.blob.core.windows.net/tools/ElasticPools/Linux/15/enableagent.sh", "isPipelinesAgent": true }, "suppressFailures": null, "type": null, "typeHandlerVersion": "1.23", "typePropertiesType": "TeamServicesAgentLinux" }

DevOpsAzurance commented 2 months ago

Since this extension does not support automatic upgrades, if I need to upgrade, how would I go about doing this without having to rebuild the VMSS resource? I am getting this when running 'az vmss extension upgrade -g MyResourceGroup -n MyScaleSet':

(OperationNotAllowed) Operation 'VirtualMachineScaleSets.ExtensionRollingUpgrade.POST' is not allowed on Virtual Machine Scale Set 'MyScaleSet'. Code: OperationNotAllowed Message: Operation 'VirtualMachineScaleSets.ExtensionRollingUpgrade.POST' is not allowed on Virtual Machine Scale Set 'E MyScaleSet'.