MethodsAndPractices / vsteam

PowerShell module for accessing Azure DevOps Services and Azure DevOps Server (formerly VSTS or TFS)
https://methodsandpractices.github.io/vsteam-docs/
MIT License
445 stars 155 forks source link

[bug] Get-VSTeamBuild throws a "NullReferenceException" on 7.0.0 runing in AzDo hosted agent #353

Closed mortenlerudjordet closed 3 years ago

mortenlerudjordet commented 3 years ago

Using 7.0.0 on a AzDo hosted agent running windows-latest (and vs2017-win2016) image Get-VSTeamBuild always crashes in an unhandled exception. Downgrading to 6.5.1 and the error goes away.

I have only managed to get the crash debugging locally one time on 7.1(p7). Have tried both 7.1 and 7.0.3 without managing to reproduce locally.

In addition I see that the object structure have changed in the return from Get-VSTeamBuild. Previous I could do $BuildInfo.triggerInfo, now this seems to be under $BuildInfo.InternalObject.triggerInfo in 7.

Steps to reproduce

$BuildID = $env:BUILD_BUILDID
Set-VSTeamAccount -Account $AzDOurl -Token $AccessToken -UseBearerToken -Version AzD
$BuildInfo = Get-VSTeamBuild -ProjectName $AzDOprojectName -Id $BuildID

Expected behavior

$BuildInfo gets populated by the info from the build. This works as expected on 6.5.1 with the same exact code.

Actual behavior

Exception calling ".ctor" with "2" argument(s): "Object reference not set to an instance of an object."

I have tracked to the following line in function Get-VSTeamBuild the one time i managed to catch it in the debugger:
$build = [vsteam_lib.Build]::new($resp, $ProjectName)
Both $resp and $ProjectName variables where not empty

Environment data

OS

Server

> Get-VSTeamAPIVersion
Name                           Value
----                           -----
Packaging                      6.0-preview
DistributedTaskReleased        5.1
DistributedTask                6.0-preview
TaskGroups                     6.0-preview
Build                          5.1
Policy                         5.1
VariableGroups                 5.1-preview.1
Release                        5.1
Git                            5.1
Processes                      6.0-preview
ServiceEndpoints               5.0-preview
Graph                          6.0-preview
MemberEntitlementManagement    6.0-preview
Core                           5.1
Tfvc                           5.1
Version                        AzD
ExtensionsManagement           6.0-preview
HierarchyQuery                 5.1-preview
> $PSVersionTable
PSVersion                      7.0.3
mortenlerudjordet commented 3 years ago

Another strong clue is that $resp seems to contain only a HTML dump. Can see AzDO Sign in page at the top of this.

SebastianSchuetze commented 3 years ago

Can you reproduce it on any project or another Org? Also stupid question: Your PAT has the right scopes?

mortenlerudjordet commented 3 years ago

@SebastianSchuetze : Yes run it on two different AzDo orgs... it's not using PAT but System.AccessToken of the hosted agent.

My testing is done with PAT locally and 19 of 20 times there is no problem. I had to run debug many times to get one that would crash, though on azure devops hosted agent it crashes each run.

image

SebastianSchuetze commented 3 years ago

I guess it is the cleaner fresh environment on the agent that makes it more "stable" or reproducable. Which is good. Because it works on your (dev) machine. 😄

mortenlerudjordet commented 3 years ago

From using previous module version with same code: image

DarqueWarrior commented 3 years ago

Thanks so much for taking the time to report. So sorry for the trouble. I will resurface triggerInfo for you and start working on a fix.

DarqueWarrior commented 3 years ago

I noticed that Trigger info is an empty object on all my tests. What were you expecting to find?

mortenlerudjordet commented 3 years ago

@DarqueWarrior : It's a feature of multi-repo trigger support in YAML build. At the moment the internal build variables are not populating with the repo and branch info of the triggering (CI) pull/merge request.

https://docs.microsoft.com/en-us/azure/devops/pipelines/repos/multi-repo-checkout?view=azure-devops

So I had to write code to fetch TriggerInfo on my own, and then write custom git checkout logic that works with the repo that triggers the build and not the repo where the build yaml resides.

DarqueWarrior commented 3 years ago

Ah! Good to know you are using YAML. I am using classic for my testing and it is still failing but I will test with YAML as well. That might populate the triggerInfo so I can test that as well.

Good news I am able to duplicate.

mortenlerudjordet commented 3 years ago

I have been using this on 6.5.1 in production for some time. With TriggerInfo and all. So we went from 20 separate build configs, down to one using multi-repo CI triggers.

Excellent you managed to reproduce.

Example from build YAML:

# No CI trigger for build YAML repo
trigger: none

# Add all c3 microservices that needs to build a container here
resources:
  repositories:
    - repository: CheckLists
      type: git
      name: "Contoso/CheckLists"
      trigger:
        branches:
          include:
            - "master"
            - "dev"
      paths:
      include:
        - "*"
      exclude:
        - "README.md"

    - repository: CRM
      type: git
      name: "Contoso/CRM"
      trigger:
        branches:
          include:
            - "master"
            - "dev"
      paths:
      include:
        - "*"
      exclude:
        - "README.md"

jobs:
  - job: job_get_trigger
    displayName: "Get Trigger Info"
    pool:
      name: Azure Pipelines
      vmImage: windows-latest
    steps:
      - task: PowerShell@2
        name: "TriggerInfo"
        displayName: "Get Trigger Build Info"
        inputs:
          targetType: filePath
          filePath: '$(System.DefaultWorkingDirectory)\AzP-GetTriggerInfo.ps1'
          arguments: "-AccessToken '$(System.AccessToken)' -AzDOprojectName '$(AzDOprojectName)'"
          pwsh: true
          errorActionPreference: "stop"
DarqueWarrior commented 3 years ago

I think I found the crash. The Queue is not set while the build is in progress.

DarqueWarrior commented 3 years ago

The fix is running through the pipeline now. Will merge and publish as soon as everything is green. Thanks again for reporting this.

DarqueWarrior commented 3 years ago

Fixed in 7.0.1 which is rolling out now.

DarqueWarrior commented 3 years ago

Hey, @mortenlerudjordet 7.0.1 is now live. Let me know if that fixed your issue for you.

mortenlerudjordet commented 3 years ago

@DarqueWarrior : No more crashes in my dev env...so seems good to go. Thanks for fixing the issue this fast.

Also exposing TriggerInfo in the original way.