Add support for Azure DevOps API-based datasource

[jessehouwing]

What would you like Renovate to be able to do?

Renovate has support for Azure-DevOps built-in tasks and the Azure DevOps Marketplace's public tasks.

But it doesn't support tasks that were installed directly into an Azure DevOps Organization (tfx build tasks upload) nor does it support extensions that are privately listed on the marketplace.

Renovate also won't be too helpful for people using Team Foundation Server or Azure DevOps Server, since these might not even have all tasks available, nor won't they have the same list of built-in tasks available to them.

Each instance of Azure DevOps however, has a set of APIs that one can query to provide grab the data needed directly from the instance. Below is a set of APIs to query to get all the information required to give proper advice

If you have any ideas on how this should be implemented, please tell us here.

• [ ] Have a way to configure the PAT to access the marketplace and the Azure DevOps Organization URI
• [ ] Potentially support Azure DevOps Server
• [ ] Query all tasks installed in an organization to get the task name's and ids (GET https://dev.azure.com/jessehouwing/_apis/distributedtask/tasks/)
  • [ ] Tasks can be identified by:
    • [ ] task.name
    • [ ] task.id
    • [ ] task.contributionIdentifier.task.name
    • [ ] task.contributionIdentifier.task.id
  • [ ] The built-in tasks can be identified by "serverOwned":true
• [ ] Query each task by id to get a list of versions (GET https://dev.azure.com/jessehouwing/_apis/distributedtask/tasks/{{task.Id}}?allversions=true
• [ ] Use the ResourceArea's API to find the ExtensionManagement API (GET https://dev.azure.com/jessehouwing/_apis/resourceareas/)
• [ ] Fetch the extention manifest from the marketplace API (GET https://extmgmt.dev.azure.com/jessehouwing/_apis/extensionmanagement/installedextensionsbyname/jessehouwing/jessehouwing-vsts-variable-tasks?api-version=7.1-preview.1&assetTypes=Microsoft.VisualStudio.Services.Manifest) using the contribution identifier task.contributionIdentifier.
• [ ] Grab the repository data (repository.uri) from the fetched manifest file (GET https://jessehouwing.gallerycdn.vsassets.io/extensions/jessehouwing/jessehouwing-vsts-variable-tasks/3.0.10/1670319234705/Microsoft.VisualStudio.Services.Manifest)

Ideally Renovate should also suggest:

• [ ] Tasks renames. These can be found by looking for a task with the same task.id, a higher task.version, but a different task.name.
• [ ] Removing deprecated tasks. These can be found by looking at the task.deprecated: true.
• [ ] Making it configurable to suggest preview versions task.preview: true.

If RenovateBot would support Private Extensions, then a few new issues pop-up, since private extensions can publish the same tasks (same task.name), even with the same version. In that case a task must be disambiguated by its task.id or by using the long name task.contributionIdentifier.task.name.

The current mechanism, with statically generated files, would allow suggestions for Azure DevOps Server in the case of a non-marketplace connected server instance where the admin would have to download and install the extension manually.

We should probably make the desired sources configurable, as renovatebot might expose the names of internal extensions and tasks through calling out to GitHub.

[jessehouwing]

@JamieMagee I found the old Azure Pipelines issue and for most of the problems with the current extension, I think I know the solution and have documented the correct APIs to use above.

• Related: https://github.com/renovatebot/renovate/issues/15818