pester / Pester

Pester is the ubiquitous test and mock framework for PowerShell.
https://pester.dev/
Other
3.11k stars 473 forks source link

Errors from tests not displaying variables #2376

Closed Leshmoe closed 6 months ago

Leshmoe commented 1 year ago

Checklist

What is the issue?


If it passes it shows correctly in the report: Edgewood - WWW.Checking Main WWW at https://www.edgewood.edu.Website Test https://www.edgewood.edu - Passed If it fails it shows incorrectly: Edgewood - WWW.Checking at .Website Test - Failed

Expected Behavior

What I would expect the output to be would be Edgewood - WWW.Checking Main WWW at https://www.edgewood.edu.Website Test https://www.edgewood.edu - Failed

Steps To Reproduce

$pesterArgs = [PesterConfiguration]::Default
$pesterArgs.Run.Path = ".\Tests\*"
$pesterArgs.Run.PassThru = $true
$pesterArgs.Output.Verbosity = "Detailed"
$pesterArgs.Should.ErrorAction = "Continue"
$pesterArgs.TestResult.Enabled = $true
$pesterArgs.TestResult.OutputPath = '.\Results\Testresult.xml'
$pesterArgs.CodeCoverage.Enabled = $true
$pesterArgs.CodeCoverage.RecursePaths = $true
$pesterArgs.CodeCoverage.OutputFormat = 'NUnit2.5'
$pesterArgs.CodeCoverage.OutputPath = ".\Results\coverage.xml"

$Results = invoke-pester -Configuration $pesterArgs
BeforeDiscovery {
    $EC_WWW_URIs = @(
        @{"Main WWW" = "https://www.edgewood.edu"}
}

Describe "Edgewood - WWW" -ForEach $EC_WWW_URIs {
    BeforeAll {
        $WebURI = $($_)
        # $URI
        $WebName = $WebURI.Keys
        $WebURL = $WebURI.Values
        $StatusCode = (Invoke-WebRequest  $($WebURL)).StatusCode 
        #Write-Host("(Invoke-WebRequest -Uri $URL -Method Get -UseDefaultCredentials).StatusCode")
         }

       Context "Checking <WebName> at <WebURL>" {
            It "Website Test <WebURL>"  { 
                $StatusCode | Should -Match "200"
            }  
        } 
}

Describe your environment

Pester version : 5.4.1 C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.1\Pester.psm1 PowerShell version : 5.1.20348.1366 OS version : Microsoft Windows NT 10.0.20348.0

Possible Solution?

No response

fflaten commented 1 year ago

Hi. Thanks for the issue. The repro doesn't fail and there's no error provided in the issue. I assume you meant when Invoke-WebRequest fails so I used a bad URL to reproduce.

In that case this is expected behavior. The failure occurs inside a setup-block (BeforeAll/-Each) which stops further processing of the block, incl. updating the names of the current block and child blocks/tests. That's because Pester can't know which variables got updated before failure.

Pester 5.4.0 improved this a bit by expanding the variables as far as possible (parent block) when there's a failure or skip. See this comment.

In case you only need the status code inside that single test, then I'd suggest moving the Invoke-WebRequest call into the test. The names are update prior to executing the code in It which means the name would be correct.

# filename: demoIssue2376.tests.ps1
BeforeDiscovery {
    # Tip: Providing hashtables to `-ForEach` will automatically create variable by the key-name
    $EC_WWW_URIs = @(
        @{ WebName = 'Main WWW'; WebURL = 'https://www.edgewood.edu' }
        @{ WebName = 'Fail WWW'; WebURL = 'https://www.edgewood.edusssss' }
    )
}

Describe 'Edgewood - <WebName>' -ForEach $EC_WWW_URIs {
    # $WebName and $WebURL exists at this point and can even be used in Describe name (just like a variable set in BeforeAll here)
    It 'Website Test <WebURL>' {
        $StatusCode = (Invoke-WebRequest $($WebURL)).StatusCode
        $StatusCode | Should -Match '200'
    }
}
> $p = invoke-pester -path /workspaces/Pester/Samples/demoIssue2376.tests.ps1 -Output None -PassThru
> $p.tests | fl ExpandedPath, Result

ExpandedPath : Edgewood - Main WWW.Website Test https://www.edgewood.edu
Result       : Passed

ExpandedPath : Edgewood - Fail WWW.Website Test https://www.edgewood.edusssss
Result       : Failed