search

Azure / azure-cli

Azure Command-Line Interface
MIT License
3.98k stars 2.96k forks source link

`az upgrade` fails on Azure DevOps Linux agents #18363

Open aheld opened 3 years ago

aheld commented 3 years ago

Describe the bug az cli on the default build server is outdated and cannot be upgraded. The version in the build agent has a know bug that blocks deployments

To Reproduce Run a pipeline on the default Azure DevOps build server

  - task: AzureCLI@2
      displayName: Try to upgrade az cli
      inputs:
        failOnStandardError: true
        azureSubscription: $(Subscription)
        scriptType: bash
        scriptLocation: inlineScript
        inlineScript: |
          az upgrade --yes

Expected behavior the az upgrade succeeds

Environment summary

### Build agent
2021-06-04T22:08:16.2093723Z ##[section]Starting: Initialize job
2021-06-04T22:08:16.2095017Z Agent name: 'Azure Pipelines 6'
2021-06-04T22:08:16.2095440Z Agent machine name: 'fv-az70-667'
2021-06-04T22:08:16.2095707Z Current agent version: '2.187.2'
2021-06-04T22:08:16.2135395Z ##[group]Operating System
2021-06-04T22:08:16.2135657Z Ubuntu
2021-06-04T22:08:16.2135832Z 16.04.7
2021-06-04T22:08:16.2135983Z LTS
2021-06-04T22:08:16.2136161Z ##[endgroup]
2021-06-04T22:08:16.2136350Z ##[group]Virtual Environment
2021-06-04T22:08:16.2136590Z Environment: ubuntu-16.04
2021-06-04T22:08:16.2136809Z Version: 20210531.0
2021-06-04T22:08:16.2137135Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu16/20210531.0/images/linux/Ubuntu1604-README.md
2021-06-04T22:08:16.2137571Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu16%2F20210531.0
2021-06-04T22:08:16.2138126Z ##[endgroup]
2021-06-04T22:08:16.2139345Z Current image version: '20210531.0'
2021-06-04T22:08:16.2153862Z Agent running as: 'vsts'
2021-06-04T22:08:16.2203634Z Prepare build directory.
2021-06-04T22:08:16.2501236Z Set build variables.
2021-06-04T22:08:16.2535532Z Download all required tasks.
2021-06-04T22:08:16.2654805Z Downloading task: AzureCLI (2.1.0)
2021-06-04T22:08:17.0162300Z Checking job knob settings.
2021-06-04T22:08:17.0171242Z    Knob: AgentToolsDirectory = /opt/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY} 
2021-06-04T22:08:17.0172612Z    Knob: AgentPerflog = /home/vsts/perflog Source: ${VSTS_AGENT_PERFLOG} 
2021-06-04T22:08:17.0175023Z    Knob: TaskRestrictionsEnforcementMode = Enabled Source: $(agent.taskRestrictionsEnforcementMode) 
2021-06-04T22:08:17.0175872Z    Knob: EnableTaskRestrictionsTelemetry = true Source: $(agent.enableTaskRestrictionsTelemetry) 
2021-06-04T22:08:17.0176688Z Finished checking job knob settings.
2021-06-04T22:08:17.0805808Z Plugin: 'Test Result Parser plugin' is running in background.
2021-06-04T22:08:17.0806186Z Plugin: 'Test File Publisher plugin' is running in background.
2021-06-04T22:08:17.0806977Z Start tracking orphan processes.
2021-06-04T22:08:17.1033468Z ##[section]Finishing: Initialize job

Upgrade Results

2021-06-04T21:44:41.5163343Z ##[section]Starting: test12 Sandbox Deploy Service Settings
2021-06-04T21:44:41.5173097Z ==============================================================================
2021-06-04T21:44:41.5173724Z Task         : Azure CLI
2021-06-04T21:44:41.5174360Z Description  : Run Azure CLI commands against an Azure subscription in a PowerShell Core/Shell script when running on Linux agent or PowerShell/PowerShell Core/Batch script when running on Windows agent.
2021-06-04T21:44:41.5175014Z Version      : 2.1.0
2021-06-04T21:44:41.5175383Z Author       : Microsoft Corporation
2021-06-04T21:44:41.5175865Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
2021-06-04T21:44:41.5176382Z ==============================================================================
2021-06-04T21:44:41.6616357Z [command]/usr/bin/az --version
2021-06-04T21:44:42.7029599Z azure-cli                         2.24.0 *
2021-06-04T21:44:42.7030438Z 
2021-06-04T21:44:42.7031018Z core                              2.24.0 *
2021-06-04T21:44:42.7031768Z telemetry                          1.0.6
2021-06-04T21:44:42.7032131Z 
2021-06-04T21:44:42.7032575Z Extensions:
2021-06-04T21:44:42.7033375Z azure-devops                      0.18.0
2021-06-04T21:44:42.7033780Z 
2021-06-04T21:44:42.7034511Z Python location '/opt/az/bin/python3'
2021-06-04T21:44:42.7035391Z Extensions directory '/opt/az/azcliextensions'
2021-06-04T21:44:42.7035813Z 
2021-06-04T21:44:42.7036367Z Python (Linux) 3.6.10 (default, May 21 2021, 07:22:47) 
2021-06-04T21:44:42.7036954Z [GCC 5.4.0 20160609]
2021-06-04T21:44:42.7037278Z 
2021-06-04T21:44:42.7037772Z Legal docs and information: aka.ms/AzureCliLegal
2021-06-04T21:44:42.7038128Z 
2021-06-04T21:44:42.7038403Z 
2021-06-04T21:44:42.7047675Z WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'
2021-06-04T21:44:42.7048776Z 
2021-06-04T21:44:42.7049555Z Please let us know how we are doing: https://aka.ms/azureclihats
2021-06-04T21:44:42.7051521Z and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
2021-06-04T21:44:42.7056539Z Setting AZURE_CONFIG_DIR env variable to: /home/vsts/work/_temp/.azclitask

2021-06-04T21:44:46.0158633Z WARNING: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
2021-06-04T21:44:46.0160306Z WARNING: Your current Azure CLI version is 2.24.0. It will be updated to 2.24.2.
2021-06-04T21:44:46.1098692Z Hit:1 http://azure.archive.ubuntu.com/ubuntu xenial InRelease
2021-06-04T21:44:46.1100718Z Get:2 http://azure.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
2021-06-04T21:44:46.1133758Z Get:3 http://azure.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
2021-06-04T21:44:46.1361403Z Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
2021-06-04T21:44:46.1772743Z Get:5 https://packages.microsoft.com/ubuntu/16.04/prod xenial InRelease [4,002 B]
2021-06-04T21:44:46.2575656Z Hit:6 http://ppa.launchpad.net/ansible/ansible/ubuntu xenial InRelease
2021-06-04T21:44:46.2974062Z Get:7 http://azure.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [2,049 kB]
2021-06-04T21:44:46.3987911Z Get:8 https://esm.ubuntu.com/infra/ubuntu xenial-infra-security InRelease [7,500 B]
2021-06-04T21:44:46.4154694Z Hit:9 http://ppa.launchpad.net/mercurial-ppa/releases/ubuntu xenial InRelease
2021-06-04T21:44:46.4184163Z Get:10 https://esm.ubuntu.com/infra/ubuntu xenial-infra-updates InRelease [7,475 B]
2021-06-04T21:44:46.4489278Z Get:11 https://packages.microsoft.com/ubuntu/16.04/prod xenial/main amd64 Packages [210 kB]
2021-06-04T21:44:46.5750744Z Get:12 http://ppa.launchpad.net/ondrej/php/ubuntu xenial InRelease [23.9 kB]
2021-06-04T21:44:46.6546375Z Get:13 https://esm.ubuntu.com/infra/ubuntu xenial-infra-security/main amd64 Packages [76.9 kB]
2021-06-04T21:44:46.8291283Z Get:14 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease [23.8 kB]
2021-06-04T21:44:47.0823779Z Get:15 http://ppa.launchpad.net/ondrej/php/ubuntu xenial/main amd64 Packages [1,260 B]
2021-06-04T21:44:47.2438757Z Get:16 http://ppa.launchpad.net/ondrej/php/ubuntu xenial/main Translation-en [1,256 B]
2021-06-04T21:44:47.4067698Z Get:17 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main amd64 Packages [88.6 kB]
2021-06-04T21:44:47.8003261Z Get:18 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main Translation-en [26.8 kB]
2021-06-04T21:44:50.2643696Z Fetched 2,845 kB in 1s (1,560 kB/s)
2021-06-04T21:44:51.1960297Z Reading package lists...
2021-06-04T21:44:51.2588165Z Reading package lists...
2021-06-04T21:44:51.4019338Z Building dependency tree...
2021-06-04T21:44:51.4027718Z Reading state information...
2021-06-04T21:44:51.5208335Z azure-cli is already the newest version (2.24.0-1~xenial).
2021-06-04T21:44:51.5542588Z 0 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.

2021-06-04T21:44:51.8136416Z WARNING: CLI upgrade failed or aborted.

2021-06-04T21:45:05.7182633Z ##[error]WARNING: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
2021-06-04T21:45:05.7193020Z ##[error]WARNING: Your current Azure CLI version is 2.24.0. It will be updated to 2.24.2.
2021-06-04T21:45:05.7194582Z ##[error]WARNING: CLI upgrade failed or aborted.
2021-06-04T21:45:05.7196430Z ##[error]Script has output to stderr. Failing as failOnStdErr is set to true.
2021-06-04T21:45:05.7202522Z [command]/usr/bin/az account clear
2021-06-04T21:45:05.9729123Z ##[section]Finishing: test12 Sandbox Deploy Service Settings
yonzhan commented 3 years ago

@jiasli for awareness

jiasli commented 3 years ago

The apt-get log provided doesn't contain 

Get:1 https://packages.microsoft.com/repos/azure-cli focal InRelease [10.4 kB]

On Azure DevOps agent, Azure CLI is not installed following official installation method, so https://packages.microsoft.com/repos/azure-cli Linux Repo is not added to apt's source list /etc/apt/sources.list.d/azure-cli.list and az upgrade doesn't work in Azure DevOps agent.

Solutions

You may either

  1. Wait for Azure DevOps agent to upgrade to the latest Azure CLI by itself (no action from your side)
  2. Install Azure CLI from the official source: https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#option-1-install-with-one-command
jiasli commented 3 years ago

But indeed, az upgrade should either

  1. Explicitly tell the user upgrading in Azure DevOps agent is not supported https://github.com/Azure/azure-cli/blob/7f89f1df5010ea5388bca09a237021415584e0b6/src/azure-cli/azure/cli/command_modules/util/custom.py#L83-L84
  2. Adding the Linux Repo by itself, but this would require additional dependencies (ca-certificates curl apt-transport-https lsb-release gnupg) to be installed
jiasli commented 1 year ago

Besides, the topic of using failOnStderr is tracked by https://github.com/Azure/azure-cli/issues/18372.