MethodsAndPractices / vsteam

PowerShell module for accessing Azure DevOps Services and Azure DevOps Server (formerly VSTS or TFS)
https://methodsandpractices.github.io/vsteam-docs/
MIT License
447 stars 155 forks source link

Test-VSTeamYamlPipeline fails if no pipeline runs on master/main exist #428

Closed hvl71 closed 2 years ago

hvl71 commented 2 years ago

Hi

We encounter an issue when trying to validate a yaml pipeline using the Test-VSTeamYamlPipeline cmdlet.

We run:

Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -FilePath C:\src\sandbox\azure-pipelines.yml

Result:

WARNING: Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference exists in the
source repository.
Test-VSTeamYamlPipeline : Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference
 exists in the source repository.
At line:1 char:1
+ Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-VSTeamYamlPipeline

We see the behaviour on both versions we have tried (7.2.0 and 7.4.0)

We have observed that our PipelineId only has pipeline runs on our development branch. We have never run the pipeline on our master branch (we don't have a main branch).

If we validate the yaml agains another PipelineId which has had a pipeline run against the relevant master branch, the yaml validates fine.

Steps to reproduce Validate a known valid yaml pipeline against a PipelineId that have no pipeline runs on master or main in Azure Devops Service

Expected behavior Yaml validates without errors or fails with more clear error message about reason

Actual behavior:

WARNING: Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference exists in the
source repository.
Test-VSTeamYamlPipeline : Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference
 exists in the source repository.
At line:1 char:1
+ Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-VSTeamYamlPipeline

Environment data: Windows 10 Server: Azure DevOps Service

Get-VSTeamAPIVersion:

Billing                     : 5.1-preview.1
Build                       : 5.1
Core                        : 5.1
DistributedTask             : 6.0-preview
DistributedTaskReleased     : 5.1
ExtensionsManagement        : 6.0-preview
Git                         : 5.1
Graph                       : 6.0-preview
HierarchyQuery              : 5.1-preview
MemberEntitlementManagement : 6.0-preview
Packaging                   : 6.0-preview
Pipelines                   : 5.1-preview
Policy                      : 5.1
Processes                   : 6.0-preview
Release                     : 5.1
ServiceEndpoints            : 5.0-preview
TaskGroups                  : 6.0-preview
Tfvc                        : 5.1
VariableGroups              : 5.1-preview.1
Version                     : VSTS
$PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.19041.1237
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.1237
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
SebastianSchuetze commented 2 years ago

Thanks for the report @hvl71 it checked it and it is per se not a bug with the cmdlet, but implemented incomplete in regards to the API.

Also documenting it for myself here. So, I had the following setup:

Following error I get:

PS F:\repos\general_github\vsteam> Set-VSTeamDefaultProject -Project 'YAML Tests'

PS F:\repos\general_github\vsteam> Test-VSTeamYamlPipeline -PipelineId 41        

WARNING: An error occurred while loading the YAML build pipeline. File /azure-pipelines.yml not found in repository https://dev.azure.com/razorspoint-test/YAML%20Tests/_git/YAML%20Tests branch refs/heads/main version 96e7436b59b550a411dc5f6c7dc42f7a5f8416bb.

Warning is different but comes down to the same problem.

When calling the following API endpoint: https://docs.microsoft.com/en-us/rest/api/azure/devops/pipelines/preview/preview?view=azure-devops-rest-6.1

You can give the pipeline Id. If you don't specify anything else, it tries to do the test against the branch which is named "main" of the repo. Additionally, I tried to switch the default branch to dev and then run it. It still didn't work.

How to resolve it?

  1. It is a bug in the API in your case (if you only have a dev branch and that is the default one) so I will report it so they may be able to fix it (e.g. take the current default branch if nothing is given)
  2. Will add a Parameter Branch to the cmdlet which creates a resources JSON object in the body like described in the API. Checked it even how the GUI does it when trying to do a validation:

image

and it sends an object like the following which is mostly empty and always seems to have only one resource in it:

{
   "resources": {
      "pipelines": {},
      "repositories": {
         "self": {
            "refName": "dev"
         }
      },
      "builds": {},
      "containers": {},
      "packages": {}
   },
   "templateParameters": {},
   "previewRun": true,
   "yamlOverride": "YAML Code here"
}
SebastianSchuetze commented 2 years ago

I reported that problem to the support: https://developercommunity.visualstudio.com/t/api-to-preview-pipeline-run-takes-non-existing-bra/1635377