Convert-ALCOutputToAzureDevOps fails on warnings

[MatDDev]

Describe the issue I am using Convert-ALCOutputToAzureDevOps cmdlet in CI pipeline to compile apps and report results of such compilation. I am using "-FailOn error" parameter, and even when I have no errors (only warnings) the build still fails (it's being reported partially succeeded, but it has status "failed" in Azure DevOps).

Scripts used to create container and cause the issue

$rulesetFile = Join-Path $appProjectFolder -ChildPath '.vscode\company.ruleset.json'
    $alcParameters = @("/project:""$($appProjectFolder.TrimEnd('/\'))""", "/packagecachepath:""$($symbolsFolder.TrimEnd('/\'))""", "/out:""$outputFolder""")
    $alcParameters += @("/analyzer:$(Join-Path "$compilerFolder\extension\bin\" 'Analyzers\Microsoft.Dynamics.Nav.UICop.dll')")
    $alcParameters += @("/analyzer:$(Join-Path "$compilerFolder\extension\bin\" 'Analyzers\Microsoft.Dynamics.Nav.CodeCop.dll')")
    $alcParameters += @("/ruleset:$rulesetfile")

    write-host "Compiling app:"
    Write-Host "$compilerPath $([string]::Join(' ', $alcParameters))"

    $compileExpression = '&{0} {1}' -f $compilerPath, $([string]::Join(' ', $alcParameters))
    $compileExpression += '; if ($lastexitcode -ne 0) {"App generation failed with exit code $lastexitcode"}'

    $result = Invoke-Expression -Command $compileExpression

    $devOpsResult = ""
    if ($result) {
        $devOpsResult = Convert-ALCOutputToAzureDevOps -AlcOutput $result -FailOn 'error' -DoNotWriteToHost
    }

Full output of scripts

BCContainerHelper is version 2.0.12
BCContainerHelper is running as administrator
Host is Microsoft Windows Server 2012 R2
...

Screenshots

Am I doing something wrong, or why does the entire pipeline seems to fail where there are no errors? If I delete the FailOn parameters, then pipeline runs successfully.

[freddydk]

Could you share the output of the compiler ($result) and the output of the function ($devOpsResult)?

[freddydk]

It should return

##vso[task.complete result=SucceededWithIssues;]Succeeded With Issues.

[MatDDev]

It does:

Compilation ended at '15:57:32.0'. ##vso[task.complete result=SucceededWithIssues;]Succeeded With Issues.

The pipeline has a warning symbol

But Azure DevOps still shows this as failed attempt. Could it be something on Azure DevOps itself?

[freddydk]

The exclamation mark is not a failed attempt - it is succeeded with issues. Here: The X means failure. In the build overview it also says warning (and you can even see the warnings in the overview): and in the build, it completes the remaining steps and publishes artifacts and all: Everything in my pipeline is as expected. These warnings are things that will turn into errors, and you should really fix them asap. If you specify -failon None it will still fail the pipeline if the app cannot build - it just means that it won't fail on the output of alc.exe.

[MatDDev]

That part is ok, but then why does it specifically say "failed" on the branch, on which the pipeline run with FailOn error, where in fact there are no errors in the run? It's just quite super confusing : )

When there's no parameter FailOn set, it all looks ok

[freddydk]

I cannot tell you why Azure DevOps has decided to have this indicator show failed when the build is successful with warnings. That would be a bug for Azure DevOps IMO.

[MatDDev]

Yeah, I kinda figured it out I need to post it on other place ^^ Thanks for the explanation and help.

Edit: Although I do have other pipelines in which I do not use Convert-ALCOutputToAzureDevOps to report warnings and these pipelines are marked as successed. I will dig into that deeper.

[freddydk]

Do these pipelines also report warnings indicated as warnings by Azure DevOps? (and SucceededWithIssues)

[MatDDev]

Do these pipelines also report warnings indicated as warnings by Azure DevOps? (and SucceededWithIssues)

Yes, although not through Convert-ALCOutputToAzureDevOps cmdlet, but Compile-AppInBcContainer. Though I can see that I don't have FailOn parameter specified there at all. Anyway, thanks for help : )

Edit: This behavior was already reported by others, but no actions have been taken I guess: https://developercommunity.visualstudio.com/t/azuredevops-shows-incorrect-pipeline-status-on-the/923077