pester / Pester

Pester is the ubiquitous test and mock framework for PowerShell.
https://pester.dev/
Other
3.11k stars 473 forks source link

Set-TestInconclusive throws error if used within 'Describe', but not 'It' block #661

Closed ymarushchenko closed 7 years ago

ymarushchenko commented 7 years ago

Hello!

My tests use It blocks inside ForEach-Object loop that receives an array of test objects as input. I want to be able to set the test status as inconclusive if this array is empty.

I can check if an array is empty within Describe block, but there is a problem with using Set-TestInconclusive there if it is not placed within It block:

at Set-TestInconclusive, C:\Program Files\WindowsPowerShell\Modules\Pester\3.4.3\Functions\Assertions\Set-TestInconclusive.ps1: line 21

How can I use Set-TestInconclusive before It block, just in Describe or Context block, where decision to run the test or not is made?

megamorf commented 7 years ago

That seems like an anti-pattern to me. Why not set all tests to inconclusive inside the foreach-loop or just skip all the tests with the following:

if ([string]::IsNullOrEmpty($yourArray))
{
    # skip tests
}
else
{
    # foreach logic
}
ymarushchenko commented 7 years ago

According to Wiki, Describe corresponds to Arrange part of the Arrange-Act-Assert approach, so making decision in this section seems reasonable to me. It corresponds to Assert part of that approach, and should only contain test-related assertions (meaning if this part is executed, the test should either pass or fail and any inconclusiveness should be excluded). In other words, I treat Describe as a section that can contain prerequisites/dependencies for the test. If all dependencies described there are present, then we can proceed to the test and make pure assertions in It block. If dependencies are not present, we just mark the test as inconclusive (all inside the Describe block, without entering It block) and proceed with the next test (the next Describe block in file).

nohwnd commented 7 years ago

This depends highly on your personal preference, and my personal preference is to put as much code as I can in the It block, and as little as I can in the Describe or anywhere else. For me the It is the AAA block.

What is your actual scenario? Are you writing tests for code? Are you validating an environment? What are the "objects" that you validate in the foreach loop?

I would be very cautious about putting any logic (not even a simple if or foreach), in my tests.

ymarushchenko commented 7 years ago

@nohwnd I have written a set of functions that I use to test my project's website. One of the functions (let's call it Get-Object here) returns PS custom objects that have multiple properties (not an array of objects, but objects one by one). My Pester scenario is to assert that these properties have correct values. Expected values are pulled from JSON file with baseline data. So, basically my test is:

  1. Run Get-Object and get objects that I need to test.
  2. Pipe these objects to ForEach-Object cmdlet.
  3. Check property values within It block inside that ForEach-Object's script block:
Describe 'Object properties in production environment' -Tags objects,production {
        Get-Object -Environment 'production' |
            ForEach-Object {
                $actual = # get actual values from objects
                $expected = # get expected values from test data JSON
                    It "$($_.ObjectTitle) property values are correct" {
                        $actual.Property1 | Should Be $expected.Property1
                        $actual.Property2 | Should Be $expected.Property2                        
                        $actual.Property3 | Should Be $expected.Property3
                    }
            }
}

It can happen that Get-Object returns nothing, so I would like to set the test inconclusive between Describe and Get-Object lines avoiding any comparisons in It block, like this:

if ((Get-Object -Environment 'production' | Measure-Object).Count -eq 0){
    Set-TestInconclusive -Message 'No objects returned, so test cannot be performed!'
}

And after that Pester should go to next Describe blocks with other tests.

it-praktyk commented 7 years ago

I assume that the issue can be closed now. Feel free to reopen if you need more assistance.