BeforeEach/BeforeAll should be able to pre-emptively fail any "It" operations

[theficus]

I do a lot of critical setup operations for my tests in BeforeEach/BeforeAll blocks. What I'm noticing is that if something fails in BeforeEach/BeforeAll that it will continue to run all of the "It" tests.

When an "It" test fails because of a BeforeEach/BeforeAll failure it's typically a real pain to troubleshoot after the fact since that history is not captured in the result output. To work around this I need to further complicate my tests by adding a bunch of assertions in the beginning to make sure my environment is in a correct state for the test to run.

This behavior seems wrong to me. I'm pretty sure MSTest doesn't work this way for comparison's sake.

Before blocks should be considered as critical (or you should be able to mark them as critical). If a Before block fails, a subsequent It should immediately fail since the state cannot be guaranteed.

Repro:

Describe "My Test" {
    BeforeEach {
        throw "I failed"
    }

    It "does something" {
        Write-Host hello
    }   
}

Output:

Describing My Test
Invoke-Blocks : I failed
At \\Tkzaw-pro-16\Mydocs8\ameltzer\My Documents\WindowsPowerShell\Modules\Pester\Functions\SetupTeardown.ps1:92 char:5
+     Invoke-Blocks -ScriptBlock $orderedSetupBlocks
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (I failed:String) [Write-Error], RuntimeException
    + FullyQualifiedErrorId : I failed,Invoke-Blocks

hello
 [+] does something 90ms

[nohwnd]

Thanks! Honestly I am bit surprised it works like that. I thought It worked like this:

try 
{
    BeforeEach
    It
}
catch 
{
    Should $_
}
finally
{
    AfterEach
}

Will have a look.

[dlwyatt]

You're right. I'm not sure what I was thinking when I originally wrote this code. The Setup / Teardown code is explicitly converting terminating errors into non-terminating (lines 131-143 of Functions/SetupTeardown.ps1), and the call to Invoke-TestCaseSetupBlocks (and similar calls) is not contained in the same Try block as the invocation of the test itself.

[nohwnd]

Got it working, figuring out the tests for it was bit of a pain. I just need to cleanup the whitespace so the Pester.Tests.ps1 stop complaining. And check if BeforeAll AfterAll follow the same rules.

[nohwnd]

Fixed by #326.