search

psake / PowerShellBuild

Common build tasks for psake and Invoke-Build that build and test PowerShell modules
MIT License
134 stars 24 forks source link

Azure DevOps builds not failing despite tests failing #31

Closed ChrisLGardner closed 5 years ago

ChrisLGardner commented 5 years ago

Expected Behavior

When a build is run via Azure DevOps I'd expect the build to fail if tests or PSSA fails. For example I have this build which failed while I was fixing various things in how it was versioning etc https://halbaradkenafin.visualstudio.com/Github/_build/results?buildId=775 and it failed correctly due to tests failing.

Current Behavior

I get builds like this one which pass despite some tests failing: https://halbaradkenafin.visualstudio.com/Github/_build/results?buildId=784

There was no real changes to the code between those two builds, I've even tried setting the ErrorActionPreference and ContinueOnError in the azure-pipelines.yml to ensure it does stop but it looks like the code isn't even getting to any of the throw statements that should cause it to stop.

ChrisLGardner commented 5 years ago

After digging into it a bit more and trying to figure out what I might have changed which broke it, turns out that I'd set an output file for the Pester results and that's causing the build to pass even if it should fail.

I can't see why that would make a difference since PassThru is always provided. I'll try to do some more digging and see if I can see what might be happening.

ChrisLGardner commented 5 years ago

And after a second look at the code for this I've spotted the issue: https://github.com/psake/PowerShellBuild/blob/6b4998bfc43f2ddb634b2c25ab42f28af196ac40/PowerShellBuild/Public/Test-PSBuildPester.ps1#L55

This should almost certainly be adding those keys to the hashtable rather than redefining the hashtable entirely. I'll throw a pull request to fix this.

devblackops commented 5 years ago

Good catch @ChrisLGardner. Thanks!