Feature: Allow users to specify custom output format

[davinci26]

Hey everyone,

Many thanks for your amazing product! I would like to propose a new feature to Pester

1. General summary of the issue

Allow users to specify a custom $OutputFormat function when invoking Pester. The invocation will look something like:

Invoke-Pester -EnableExit -OutputFormat 'Custom' -OutputFunction Export-UserDefinedReport

2. Possible Solution

This feature will allow the user to specify a custom output format. This can be achieved with the following changes:

1. Expand the outputformat parameter to also accept the literal "Custom". The outputformat parameter will default to NUnitXml (same as now)
2. Add an additional input parameter $OutputFunction that allows the user to specify the logging function. The logging function should follow the same contract as the NUnit function: Export-UserDefinedReport -PesterState $PesterState -Path $Path
3. Modify function Export-PesterResults to call the user provided function

   3. Context

   This feature is useful for users that are required to use an output format that is either proprietary or too specific to be added to Pester.

4. Implementation

I would love to hear your feedback! When the design is somewhat finalized I can go ahead and implement the feature.

[dlwyatt]

I suspect you can already accomplish this by using the -PassThru parameter, and then sending that object on to your function. That would be more resilient to changes in Pester’s internals as well.

[nohwnd]

@davinci26 yes, what @dlwyatt says is true. You can get the result object from Pester by specifying the -PassThru parameter to Invoke-Pester. This object is also used internally to produce the Nunit output. Here you can see that the $pester object is passed into a function and transformed to the NUnit XML, and here the same object is returned when -PassThru is specified. So the functionality can be implemented both internally and externally. (Some internal properties are filtered out from the object when it's published, but hopefully nothing you need for your output.)

Please give it a try and if you need more help re-open this issue.

[davinci26]

@dlwyatt @nohwnd Thanks a lot for the support :). For now this seems to be solving my problem.

[davinci26]

Hey again,

Hopefully I am not becoming annoying :) with my Issues.

I am running on the following issue when using the object generated by PassThru. The reason for that is that on failing cases I would prefer having the information that is used to create the StackTrace string in addition to the StackTrace. More specifically I would love to have the following information on Pester Output:

• Filename
• Line

This information is already available and they are used to create the StackTrace. I could use regex to parse the StackTrace and generate the fields that I want but it feels like a hacky solution.

Is there any way in Pester/PS way to retrieve these information safely without relying on regex?

[nohwnd]

Hello, no you are not. :)

The StackTrace you are looking at is the one that we format for the screen output, but there also ErrorRecord where you find complete information about the failure. Including InvocationInfo that has ScriptLineNumber, and the ScriptName, among other things.

[davinci26]

I tried your solution and it seemed correct but there were some issues there. Something feels with the ErrorRecord value in the TestResult

I have the following modified version of Examples/Calculator/Add-Numbers.Tests.ps1:

$here = Split-Path -Parent $MyInvocation.MyCommand.Path
. "$here\Add-Numbers.ps1"
Describe -Tags "Example" "Add-Numbers" {
    It "adds positive numbers" {
        Add-Numbers 2 3 | Should -Be 4
    }
}

And I execute the following commands to invoke the test

>$PesterOutput = Invoke-Pester -Show None -PassThru
>$PesterOutput.TestResult[0].ErrorRecord
Expected 4, but got 5.
At Pester/Functions/Assertions/Should.ps1:183 char:9
+         throw ( New-ShouldErrorRecord -Message $testResult.FailureMes ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidResult: (System.Collections.Hashtable:Hashtable) [], Exception
+ FullyQualifiedErrorId : PesterAssertionFailed
>$PesterOutput.TestResult[0].StackTrace
at <ScriptBlock>, Pester/Examples/Calculator/Add-Numbers.Tests.ps1: line 7
7:         Add-Numbers 2 3 | Should -Be 4

$PesterOutput.TestResult[0].ErrorRecord.InvocationInfo

MyCommand             :
BoundParameters       : {}
UnboundArguments      : {}
ScriptLineNumber      : 183
OffsetInLine          : 9
HistoryId             : -1
ScriptName            : Pester/Functions/Assertions/Should.ps1
Line                  :         throw ( New-ShouldErrorRecord -Message $testResult.FailureMessage -File $file -Line $lineNumber -LineText
                        $lineText )

PositionMessage       : At Pester/Functions/Assertions/Should.ps1:183 char:9
                        +         throw ( New-ShouldErrorRecord -Message $testResult.FailureMes ...
                        +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot          : Pester/Functions/Assertions
PSCommandPath         : Pester/Functions/Assertions/Should.ps1
InvocationName        :
PipelineLength        : 0
PipelinePosition      : 0
ExpectingInput        : False
CommandOrigin         : Internal
DisplayScriptPosition :

Shouldnt $PesterOutput.TestResult[0].ErrorRecord contain the file of the test that failed similar to $PesterOutput.TestResult[0].StackTrace ?

[nohwnd]

Okay my bad, I was testing this by throwing directly and without using Should. You probably have to stick with parsing the result. (or parsing the ScriptStackTrace) and finding a first line that points to your script and not pester internals.

[nohwnd]

Just noting that Pester v5 object is much richer and includes the formatted output on each error as well, and hopefully all data that are used to produce the output so you an re-construct it in a any way you want just by post-processing the result object.