microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.5k stars 2.61k forks source link

[REGRESSION]: FuncToolsInstaller fails when using version "latest" #20642

Open mvldk opened 2 days ago

mvldk commented 2 days ago

New issue checklist

Task name

FuncToolsInstaller

Breaking task version

0

Last working task version

0

Regression Description

Azure function core tools seems to have changed their release download file structure - notice the added "v" before the version number in the folder:

https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6594/Azure.Functions.Cli.linux-x64.4.0.6594.zip https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.4.0.6543.zip https://github.com/Azure/azure-functions-core-tools/releases/download/4.0.6280/Azure.Functions.Cli.linux-x64.4.0.6280.zip https://github.com/Azure/azure-functions-core-tools/releases/download/4.0.5907/Azure.Functions.Cli.linux-x64.4.0.5907.zip https://github.com/Azure/azure-functions-core-tools/releases/download/4.0.5858/Azure.Functions.Cli.linux-x64.4.0.5858.zip

As a result, FuncToolsInstaller fails to download the tools when configured with version "latest": Error: Unexpected HTTP response: 404 (see logs)

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

ubuntu-22.04

Relevant log output

Starting: Install Azure Func Core Tools
==============================================================================
Task         : Install Azure Func Core Tools
Description  : Install Azure Func Core Tools
Version      : 0.246.3
Author       : Microsoft Corporation
Help         : https://aka.ms/func-tools-installer
==============================================================================
Downloading: https://github.com/Azure/azure-functions-core-tools/releases/download/4.0.6543/Azure.Functions.Cli.linux-x64.4.0.6543.zip
##[error]Error: Failed to download func tools from location https://github.com/Azure/azure-functions-core-tools/releases/download/4.0.6543/Azure.Functions.Cli.linux-x64.4.0.6543.zip. Error Error: Unexpected HTTP response: 404
    at Object.<anonymous> (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:216:31)
    at Generator.next (<anonymous>)
    at fulfilled (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  httpStatusCode: 404
}.
Finishing: Install Azure Func Core Tools

Full task logs with system.debug enabled

2024-11-11T09:53:49.8199870Z ##[debug]Evaluating condition for step: 'Install Azure Func Core Tools'
2024-11-11T09:53:49.8201016Z ##[debug]Evaluating: SucceededNode()
2024-11-11T09:53:49.8201353Z ##[debug]Evaluating SucceededNode:
2024-11-11T09:53:49.8201894Z ##[debug]=> True
2024-11-11T09:53:49.8202265Z ##[debug]Result: True
2024-11-11T09:53:49.8202648Z ##[section]Starting: Install Azure Func Core Tools
2024-11-11T09:53:49.8208492Z ==============================================================================
2024-11-11T09:53:49.8208654Z Task         : Install Azure Func Core Tools
2024-11-11T09:53:49.8208760Z Description  : Install Azure Func Core Tools
2024-11-11T09:53:49.8208876Z Version      : 0.246.3
2024-11-11T09:53:49.8208961Z Author       : Microsoft Corporation
2024-11-11T09:53:49.8209060Z Help         : https://aka.ms/func-tools-installer
2024-11-11T09:53:49.8209181Z ==============================================================================
2024-11-11T09:53:49.8836869Z ##[debug]Using node path: /home/ubuntu/externals/node20_1/bin/node
2024-11-11T09:53:49.9570417Z ##[debug]agent.TempDirectory=/home/ubuntu/_work/_temp
2024-11-11T09:53:49.9579925Z ##[debug]loading inputs and endpoints
2024-11-11T09:53:49.9583883Z ##[debug]loading INPUT_VERSION
2024-11-11T09:53:49.9601175Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2024-11-11T09:53:49.9603082Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2024-11-11T09:53:49.9605391Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2024-11-11T09:53:49.9609589Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2024-11-11T09:53:49.9610751Z ##[debug]loaded 5
2024-11-11T09:53:49.9615396Z ##[debug]Agent.ProxyUrl=undefined
2024-11-11T09:53:49.9617189Z ##[debug]Agent.CAInfo=undefined
2024-11-11T09:53:49.9617601Z ##[debug]Agent.ClientCert=undefined
2024-11-11T09:53:49.9618001Z ##[debug]Agent.SkipCertValidation=undefined
2024-11-11T09:53:49.9810167Z ##[debug]Agent.ProxyUrl=undefined
2024-11-11T09:53:49.9810677Z ##[debug]Agent.CAInfo=undefined
2024-11-11T09:53:49.9811080Z ##[debug]Agent.ClientCert=undefined
2024-11-11T09:53:49.9811539Z ##[debug]check path : /home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/lib.json
2024-11-11T09:53:49.9822100Z ##[debug]adding resource file: /home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/lib.json
2024-11-11T09:53:49.9822719Z ##[debug]system.culture=en-US
2024-11-11T09:53:49.9839272Z ##[debug]check path : /home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/task.json
2024-11-11T09:53:49.9840087Z ##[debug]adding resource file: /home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/task.json
2024-11-11T09:53:49.9840549Z ##[debug]system.culture=en-US
2024-11-11T09:53:49.9844711Z ##[debug]SYSTEM_JOBID=df9359ee-092f-537f-2e8c-54834881ad22
2024-11-11T09:53:50.0084499Z ##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=FuncToolsInstallerV0]{"jobId":"df9359ee-092f-537f-2e8c-54834881ad22"}
2024-11-11T09:53:50.0085197Z ##[debug]version=latest
2024-11-11T09:53:50.0085456Z Finding latest func tools version...
2024-11-11T09:53:50.0085759Z ##[debug]
2024-11-11T09:53:50.0086108Z ##[debug]Agent.Version=3.246.0
2024-11-11T09:53:50.0086483Z ##[debug]Agent.TempDirectory=/home/ubuntu/_work/_temp
2024-11-11T09:53:50.0086856Z ##[debug]testing directory '/home/ubuntu/_work/_temp'
2024-11-11T09:53:50.0087505Z Downloading: https://api.github.com/repos/Azure/azure-functions-core-tools/releases/latest
2024-11-11T09:53:50.0087893Z ##[debug]destination /home/ubuntu/_work/_temp/d545a3b4-6b5f-4ef4-9e4d-418e88c568da
2024-11-11T09:53:50.0088275Z ##[debug]downloading
2024-11-11T09:53:50.2833471Z ##[debug]Content-Length header missing
2024-11-11T09:53:50.2833982Z ##[debug]creating stream
2024-11-11T09:53:50.6172635Z ##[debug]download complete
2024-11-11T09:53:50.6173351Z ##[debug]Downloaded file size: 32675 bytes
2024-11-11T09:53:50.6180950Z Latest version is v4.0.6543
2024-11-11T09:53:50.6186225Z ##[debug]isExplicit: 4.0.6543
2024-11-11T09:53:50.6187820Z ##[debug]explicit? true
2024-11-11T09:53:50.6188482Z ##[debug]Agent.Version=3.246.0
2024-11-11T09:53:50.6189716Z ##[debug]Agent.ToolsDirectory=/opt/hostedtoolcache
2024-11-11T09:53:50.6190226Z ##[debug]checking cache: /opt/hostedtoolcache/func/4.0.6543/x64
2024-11-11T09:53:50.6190956Z ##[debug]not found
2024-11-11T09:53:50.6195161Z ##[debug]
2024-11-11T09:53:50.6196025Z ##[debug]Agent.Version=3.246.0
2024-11-11T09:53:50.6196577Z ##[debug]Agent.TempDirectory=/home/ubuntu/_work/_temp
2024-11-11T09:53:50.6197064Z ##[debug]testing directory '/home/ubuntu/_work/_temp'
2024-11-11T09:53:50.6198062Z Downloading: https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.v4.0.6543.zip
2024-11-11T09:53:50.6198607Z ##[debug]destination /home/ubuntu/_work/_temp/b2d18200-13b2-4b56-99cd-5b53d9506864
2024-11-11T09:53:50.6199078Z ##[debug]downloading
2024-11-11T09:53:50.8577879Z ##[debug]Failed to download "b2d18200-13b2-4b56-99cd-5b53d9506864" from "https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.v4.0.6543.zip". Code(404) Message(Not Found)
2024-11-11T09:53:50.8602010Z ##[debug]task result: Failed
2024-11-11T09:53:50.8647211Z ##[error]Error: Failed to download func tools from location https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.v4.0.6543.zip. Error Error: Unexpected HTTP response: 404
    at Object.<anonymous> (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:216:31)
    at Generator.next (<anonymous>)
    at fulfilled (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  httpStatusCode: 404
}.
2024-11-11T09:53:50.8655302Z ##[debug]Processed: ##vso[task.issue type=error;]Error: Failed to download func tools from location https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.v4.0.6543.zip. Error Error: Unexpected HTTP response: 404
    at Object.<anonymous> (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:216:31)
    at Generator.next (<anonymous>)
    at fulfilled (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  httpStatusCode: 404
}.
2024-11-11T09:53:50.8657651Z ##[debug]Processed: ##vso[task.complete result=Failed;]Error: Failed to download func tools from location https://github.com/Azure/azure-functions-core-tools/releases/download/v4.0.6543/Azure.Functions.Cli.linux-x64.v4.0.6543.zip. Error Error: Unexpected HTTP response: 404
    at Object.<anonymous> (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:216:31)
    at Generator.next (<anonymous>)
    at fulfilled (/home/ubuntu/_work/_tasks/FuncToolsInstaller_a22a6eb4-fe21-442e-abce-5b2b09cfece3/0.246.3/node_modules/azure-pipelines-tool-lib/tool.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  httpStatusCode: 404
}.
2024-11-11T09:53:55.7212858Z ##[debug]Agent environment resources - Disk: / Available 17211.13 MB out of 80063.40 MB, Memory: Used 1852.00 MB out of 15988.00 MB, CPU: Usage 0.24%
2024-11-11T09:54:00.7225461Z ##[debug]Agent environment resources - Disk: / Available 17211.13 MB out of 80063.40 MB, Memory: Used 1850.00 MB out of 15988.00 MB, CPU: Usage 1.22%
2024-11-11T09:54:05.7228072Z ##[debug]Agent environment resources - Disk: / Available 17211.13 MB out of 80063.40 MB, Memory: Used 1850.00 MB out of 15988.00 MB, CPU: Usage 3.67%
2024-11-11T09:54:10.7218958Z ##[debug]Agent environment resources - Disk: / Available 17211.13 MB out of 80063.40 MB, Memory: Used 1849.00 MB out of 15988.00 MB, CPU: Usage 0.25%
2024-11-11T09:54:16.8378115Z ##[debug]Agent environment resources - Disk: / Available 17211.13 MB out of 80063.40 MB, Memory: Used 1849.00 MB out of 15988.00 MB, CPU: Usage 0.48%
2024-11-11T09:54:20.8666455Z ##[section]Finishing: Install Azure Func Core Tools

Repro steps

- task: FuncToolsInstaller@0
  displayName: Install Azure Func Core Tools
  inputs:
    version: latest
Ehsanshiri commented 2 days ago

There is also a related issue in azure-functions-core-tools repo. Indeed this could be due to a change in the versioning pattern that includes an additional prepended 'v'.

mvldk commented 2 days ago

Unless the file structure for azure function core tools is reverted, I think that the problem needs to be fixed in the pipeline task. The pipeline task contains logic to build the download URL, which would no longer work:

function getDownloadUrl(version: string) {
    let downloadUrlFormat = 'https://github.com/Azure/azure-functions-core-tools/releases/download/%s/Azure.Functions.Cli.%s.%s.zip';
    switch (os.type()) {
        case 'Linux':
            return util.format(downloadUrlFormat, version, 'linux-x64', version);

        case 'Darwin':
            return util.format(downloadUrlFormat, version, 'osx-x64', version);

        case 'Windows_NT':
        default:
            return util.format(downloadUrlFormat, version, 'win-x86', version);

    }
}

Source: https://github.com/microsoft/azure-pipelines-tasks/blob/faa494f9546c28d147fe10e986a8ad826baf2bb2/Tasks/FuncToolsInstallerV0/src/utils.ts#L95C1-L109C2

The same version string, neither "v4.0.6543" nor "4.0.6543", will work in both places of the download URL, 'https://github.com/Azure/azure-functions-core-tools/releases/download/%s/Azure.Functions.Cli.%s.%s.zip'. The "v" prefix is necessary in the first location but shouldn't be there in the second location.

Ehsanshiri commented 2 days ago

For now, I made it work for my pipeline by forcing it to use the older version but this need to be resolved:

          - task: FuncToolsInstaller@0
            displayName: 'Install Azure Functions Core Tools'
            inputs:
              version: '4.0.6280'
mnemcik-visma commented 1 day ago

Maybe change the pipeline task to use the link provided in the feed as browser_download_url? That one is set to the correct value. https://api.github.com/repos/Azure/azure-functions-core-tools/releases/latest

mvldk commented 1 day ago

@mnemcik-visma It's a bit more tricky than that. There are several assets in the release bundle, each with a specified browser_download_url. Code would have to parse asset properties to find the one relevant for the OS type.

rrr-michael-aquilina commented 1 day ago

Same issue affecting a build pipeline on my end.

Image

mnemcik-visma commented 1 day ago

@mnemcik-visma It's a bit more tricky than that. There are several assets in the release bundle, each with a specified browser_download_url. Code would have to parse asset properties to find the one relevant for the OS type.

That is exactly what I'd do by design rather than create a strong dependency on file naming conventions defined in a 3rd party repository.

v-schhabra commented 1 day ago

Hi @mvldk @Ehsanshiri @rrr-michael-aquilina @mnemcik-visma Could you please check if still you are facing the issues? I created a new release with FunctoolsInstaller task but I am unable to reproduce the issue with the repro steps shared above.

Ehsanshiri commented 1 day ago

Hi @v-schhabra Now it's working fine for the latest version. There has been an update for it here: https://github.com/Azure/azure-functions-core-tools/issues/4156#issuecomment-2469230570

mnemcik-visma commented 1 day ago

Works correctly for me with version 4.0.6594 EDIT: Kind of works correctly, as there seems to be a problem with this version of Azure Functions Tools, we're getting error when calling func start:
Failed to start the inproc6 model host. An error occurred trying to start process '/opt/hostedtoolcache/func/4.0.6594/x64/in-proc6/func' with working directory '/home/vsts/work/1/s/Service/src/Api/bin/output'. Permission denied

Reverted back to 4.0.6280, all works fine with this version.

mvldk commented 23 hours ago

Since they changed the release file structure back in Azure/azure-functions-core-tools, the FuncToolsInstaller task also works for me using latest version - but version 4.0.6543 still fails.

I would still suggest to follow through on the suggestion from @mnemcik-visma to rely on the release meta data instead of fragile naming convention:

Maybe change the pipeline task to use the link provided in the feed as browser_download_url? That one is set to the correct value. https://api.github.com/repos/Azure/azure-functions-core-tools/releases/latest

I do get the same error as @mnemcik-visma with version 4.0.6594. But I am not sure if that is related to this issue:

Kind of works correctly, as there seems to be a problem with this version of Azure Functions Tools, we're getting error when calling func start: Failed to start the inproc6 model host. An error occurred trying to start process '/opt/hostedtoolcache/func/4.0.6594/x64/in-proc6/func' with working directory '/home/vsts/work/1/s/Service/src/Api/bin/output'. Permission denied