Run.SkipRemainingOnFailure processes tests that are both excluded by filter and/or skipped using -Skip. They are included in the summary
In the repro below, 3 tests were included by filter but one was skipped. The summary says Remaining tests skipped after first failure: 3 when only 1 test was actually skipped by the plugin.
Pester v5.6.0-beta1
Starting discovery in 1 files.
Discovery found 4 tests in 18ms.
Filter 'Tag' set to ('Demo').
Filters selected 3 tests to run.
Running tests.
Describing A
This is test 2
[-] Test 1 11ms (4ms|7ms)
Expected 2, but got 1.
at 1 | Should -Be 2, untitled:Untitled-7:5
at <ScriptBlock>, untitled:Untitled-7:5
Describing B
[!] Test 3 6ms (0ms|6ms)
[!] Test 4 0ms (0ms|0ms)
Tests completed in 67ms
Tests Passed: 0, Failed: 1, Skipped: 2, Inconclusive: 0, NotRun: 1
Remaining tests skipped after first failure: 3
Expected Behavior
Don't mark excluded tests as Skip.
Update plugin summary to only count skipped tests that it was responsible for (exclude -Skip tests).
Steps To Reproduce
$sb = {
Describe 'A' {
It 'Test 1' -Tag 'Demo' {
Write-Host 'This is test 2'
1 | Should -Be 2
}
It 'Test 2' {
Write-Host 'This is test 3'
}
Describe 'B' {
It 'Test 3' -Tag 'Demo' {
Write-Host 'This is test 2'
}
It 'Test 4' -Tag 'Demo' -Skip {
Write-Host 'This is test 3'
}
}
}
}
$conf = New-PesterConfiguration
$conf.Output.Verbosity = 'Detailed'
$conf.Run.ScriptBlock = $sb
$conf.Run.SkipRemainingOnFailure = 'Block'
$conf.Filter.Tag = 'Demo'
Invoke-Pester -Configuration $conf
Describe your environment
Pester version : 5.6.0-beta1 /workspaces/Pester/bin/Pester.psm1
PowerShell version : 7.4.1
OS version : Unix 5.15.133.1
Possible Solution?
Update criteria from if (-not $test.Executed) to if ($test.ShouldRun -and -not $test.Skip -and -not $test.Executed).
Checklist
What is the issue?
Run.SkipRemainingOnFailure
processes tests that are both excluded by filter and/or skipped using-Skip
. They are included in the summaryIn the repro below, 3 tests were included by filter but one was skipped. The summary says
Remaining tests skipped after first failure: 3
when only 1 test was actually skipped by the plugin.Expected Behavior
Don't mark excluded tests as Skip. Update plugin summary to only count skipped tests that it was responsible for (exclude
-Skip
tests).Steps To Reproduce
Describe your environment
Pester version : 5.6.0-beta1 /workspaces/Pester/bin/Pester.psm1
PowerShell version : 7.4.1 OS version : Unix 5.15.133.1
Possible Solution?
Update criteria from
if (-not $test.Executed)
toif ($test.ShouldRun -and -not $test.Skip -and -not $test.Executed)
.