V5 loops in describe blocks question

[Stephanevg]

Hi guys, it might be an obivous question perhaps, but I understood that for v5 pester scripts, all the code needs to be englobed in a Beforeall / BeforeEach block, and that nothing could be set directly in the Describe block as it would result in beeing launched at discovered.

What would be the recommend ways for scenarious where loops are involved around IT blocks.

Example

Describe "Testing Disks" {

BeforeAll{
$MinimumFreeSpaceSystemDrive = 20
MinimumFreeSpaceRegularDrive = 5
$DiskData = Get-DiskData

}

foreach($Disk in $DiskData){
    if($($Disk.DeviceID) -eq $($env:SystemDrive)){
      it "[$($Disk.DeviceID)] SystemDrive Should have $($MinimumFreeSpaceSystemDrive) GB freespace"{
        Assert-DriveNotFull -DriveLetter $Disk.DeviceID -DesiredFreeSpace $MinimumFreeSpaceSystemDrive | -Should -be $True -Because "The drive $($SystemDrive) must have more than $($MinimumFreeSpaceSystemDrive) free space available."
      }
    }else{
      it "[$($Disk.DeviceID)] DataDrive Should have $($MinimumFreeSpaceRegularDrive) GB freespace"{
        Assert-DriveNotFull -DriveLetter $Disk.DeviceID -DesiredFreeSpace $MinimumFreeSpaceSystemDrive | -Should -be $True -Because "The drive $($SystemDrive) must have more than $($MinimumFreeSpaceRegularDrive) free space available."
      }
    }
  }

}#EndDescribe

Any suggestions on how should rewrite my foreach to be v5 compatible?

[nohwnd]

Probably the best option here is to run the code in BeforeAll during discovery, because I don't have test generators in place yet. And attach the data to the test as a single TestCase.

(I am using Write-Host because you have a bunch of syntax errors there, and the $SystemDrive variable does not exist, and I just want to show you that the data are available in the test.)

Describe "Testing Disks" {

    $MinimumFreeSpaceSystemDrive = 20
    $MinimumFreeSpaceRegularDrive = 5
    $DiskData = Get-DiskData

    foreach($Disk in $DiskData){
        It "[$($Disk.DeviceID)] DataDrive Should have $($MinimumFreeSpaceRegularDrive) GB freespace" -TestCases @(
            @{
                SystemDrive = $SystemDrive
                MinimumFreeSpaceSystemDrive = $MinimumFreeSpaceSystemDrive
                Disk = $Disk
            }
        ) {
            Write-Host "Assert-DriveNotFull -DriveLetter $($Disk.DeviceID) -DesiredFreeSpace $($MinimumFreeSpaceSystemDrive)" -ForegroundColor Cyan
        }
    }
}

[markgar]

Maybe I'm not understanding, but I'm not finding this syntax works. No matter what I do, when I move into the It block, any variable I set outside, for instance in the foreach statement setting the iterator, is not available inside the It block.

Meaning in your above example $Disk would be null as soon as I move into the It statement.

The It block seems to be a completely new scope with only variables available from the BeforeAll block. This is good except if you need to foreach through an array which is created in the BeforeAll block.

[nohwnd]

That is true, that is why you are not supposed to put code in Describe / Context unless you know what you are doing. What you are missing probably is that the above code is using a single -TestCases test case. This will attach the provided hashtable to the test object, and will define every key in that hashtable as a variable in the It block. So the data that were in $Disk during discovery will be again defined as $Disk in It.

[javydekoning]

I have the same or a similar question. In v4 this was fine:

$Foo = 'Bar'

Describe "DescribeName $Foo" {
    Context "ContextName $Foo" {
        It "ItName $Foo" {
            $Foo | Should -Be 'Bar'
        }
    }
}

However, in v5 $Foo no longer leaks to the it block. If we wrap the declaration of $Foo in a Before* block, it's no longer accessible outside the it block.

TestCases partially helps because I can use <Foo> in the it Name, but that doesn't work for nested things like <Sub.Name> (see below).

@{
  Name='Foo'
  Sub= @{
    Name='Bar'
  }
}

Now I can think of multiple ways to work around this, like flattening the object. But none of them seem nice and I'm trying to keep things DAMP.

Any recommendations on best practices? A simple example that works is below, but again for more complex objects that doesn't work well for me.

Describe 'PSScriptAnalyzer analysis' {    
    It "<Path> Should not return any violation for the rule : <IncludeRule>" -TestCases @(
        Foreach ($m in $Modules) {
            Foreach ($r in (Get-ScriptAnalyzerRule)) {
                @{
                    IncludeRule = $r.RuleName
                    Path        = $m.FullName
                }
            }
        }
    ) {
        param($IncludeRule, $Path)
        Invoke-ScriptAnalyzer -Path $Path -IncludeRule $IncludeRule |
        Should -BeNullOrEmpty
    }
}

[nohwnd]

@javydekoning I was considering to allow the dot notation for a while, and expand the name in the context of the It, which would make this work when you define the variable in BeforeAll. Implementing this should not be very difficult. Do you want to PR it?

[javydekoning]

Hi @nohwnd, I'm not sure what the best way would be to address that here

That said, this would still not allow access to the value outside of the It block. Like for example here:

Context "ContextName $Foo" {}

[jeremymcgee73]

Thanks for the change! I've been fighting with this for an hour or so 😊

[markgar]

@javydekoning - maybe I'm just new so I don't understand, but can you explain what you are linking us to in your comment?

[javydekoning]

@markgar I'm pointing towards the code that should get updated if you'd like to support the dot notation for more complex objects.

Consider the following object:

$obj = @{
  Name='Foo'
  Sub= @{
    Name='Bar'
  }
}

The following it block would work:

It "<Name> should ..." -TestCases $obj
}

But this does not:

It "<Sub.Name> should ..." -TestCases $obj

---

Why? In Pester v4 you could declare complex objects and loop over them, while still accessing them in Context and It blocks.

In v5 this no longer is possible. You can only do it using testcases or via the Before* blocks, but that is not accessible outside the It blocks.

For example in v4 you could do this:

[hashtable[]]$Foo = @{
    phase = 'Bar'
    attr  = @{
        name = 'Baz'
    }
}

Describe "DescribeName" {
    Foreach ($f in $Foo) {
        Context "ContextName $($f.phase)" {
            It "$($f.phase) has name $($f.attr.name) and should match Bar|Baz" {
                $f.attr.name | Should -Match 'Bar|Baz'
            }
        }
    }
}

I used to use such scenarios a lot. Maybe it's not best practice, but it makes me wonder what the best approach in v5 would be. Allowing the dot notation in 'It' blocks would partially resolve my issue and allow me to move to TestCases.

[Omzig]

I have the same problem but more complex.

I need to loop the Context and loop the It.

Describe "DescribeName" {
    Foreach ($f in $Foo) {
        Context "ContextName $($f.phase)" {
            foreach($rec in $f.subItems) {
                      It "$($f.phase) has name $($rec.name) and should match Bar|Baz" {
                           $rec.name | Should -Match 'Bar|Baz'
                       }
            }
        }
    }
}

[Omzig]

Probably the best option here is to run the code in BeforeAll during discovery, because I don't have test generators in place yet. And attach the data to the test as a single TestCase.

(I am using Write-Host because you have a bunch of syntax errors there, and the $SystemDrive variable does not exist, and I just want to show you that the data are available in the test.)

Describe "Testing Disks" {

    $MinimumFreeSpaceSystemDrive = 20
    $MinimumFreeSpaceRegularDrive = 5
    $DiskData = Get-DiskData

    foreach($Disk in $DiskData){
        It "[$($Disk.DeviceID)] DataDrive Should have $($MinimumFreeSpaceRegularDrive) GB freespace" -TestCases @(
            @{
                SystemDrive = $SystemDrive
                MinimumFreeSpaceSystemDrive = $MinimumFreeSpaceSystemDrive
                Disk = $Disk
            }
        ) {
            Write-Host "Assert-DriveNotFull -DriveLetter $($Disk.DeviceID) -DesiredFreeSpace $($MinimumFreeSpaceSystemDrive)" -ForegroundColor Cyan
        }
    }
}

This doesn't always work. I have had to convert the objects to a hash table before i pass it in. :( Ugg, idk, maybe maybe not.... I have had to rework so much of my tests because of this, i am at the point i don't know what is up or down. lol I do agree that allowing <IncludeRule> to dot <IncludeRule.Name> into the properties would be extremely helpful.

I have had more success with doing this:

Describe "Testing Disks" {

    $MinimumFreeSpaceSystemDrive = 20
    $MinimumFreeSpaceRegularDrive = 5
    $DiskData = Get-DiskData

    foreach($Disk in $DiskData){
          $script:Disk = $Disk #<<<<<<<<<<<<<<<<<<<<< minor modification.....................
        It "[$($Disk.DeviceID)] DataDrive Should have $($MinimumFreeSpaceRegularDrive) GB freespace" -TestCases @(
            @{
                SystemDrive = $SystemDrive
                MinimumFreeSpaceSystemDrive = $MinimumFreeSpaceSystemDrive
                Disk = $Disk
            }
        ) {
            Write-Host "Assert-DriveNotFull -DriveLetter $($Disk.DeviceID) -DesiredFreeSpace $($MinimumFreeSpaceSystemDrive)" -ForegroundColor Cyan
        }
    }
}

Well, i take that back, it doesn't always work, passing an assert cmlet with the contents of an IT, fails. I will have to convert this one to a hash table and see if that helps. So what happens is when the IT is run, it only uses the last value in the foreach loop :(

$appPools = Get-AppPool
foreach($pool in $appPools) {
        $script:pool = $pool

        context " - $($pool.Name) Application pool." {
                Assert-AppPoolLogonAsServiceRights -UserName (Get-AppPoolUserName $pool) -here $here
        }    
    }

If i miss one test, i could get a false positive or a false neg. :(

[efie45]

I don't mind moving the setup around for v5 but I am also running into a related issue that I see costing a lot of setup time and I'm not sure how to resolve. I am running similar tests across many different files. Previously I could dot source the script, run all the tests for that file, and then continue. Now, I need to dot source the files inside the It blocks with the filenames given from the test cases. This could mean potentially hundreds or thousands of additional sourcing calls. Is there any way around this in v5?

So I used to be able to do something like this:

$files = 'file1.ps1', 'file2.ps1', 'file3.ps1' 
$scenarios = @{inputValue = 1; expected = $true }, @{inputValue = 3; expected = $false }
@{inputValue = 4; expected = $false }, @{inputValue = 5; expected = $false }

Describe "Fake Tests" {
    foreach ($file in $files) {
        # source file once for all these tests
        . "$file"
        It "$file : <inputValue> is an integer" -TestCases $scenarios {
            $inputValue | Should -BeOfType [int]
        }
        It "$file : <inputValue> should return <expected>" -TestCases $scenarios {
            ($inputValue -eq 1) | Should -eq $expected
        }
    }
}

...which would source three files and now it seems to need to be something like this:

$files = @{file = 'file1.ps1' }, @{file = 'file2.ps1' }, @{file = 'file3.ps1' }
$scenarios = @{inputValue = 1; expected = $true }, @{inputValue = 3; expected = $false }
@{inputValue = 4; expected = $false }, @{inputValue = 5; expected = $false }
[System.Collections.ArrayList]$testCases = @()
foreach ($file in $files) {
    foreach ($scenario in $scenarios) {
        $testCases.Add($file + $scenario)
    }
}
Describe "Fake Tests" {
    It "<file> : <inputValue> is an integer" -TestCases $testCases {
        . "$file"
        $inputValue | Should -BeOfType [int]
    }
    It "<file> : <inputValue> should return <expected>" -TestCases $testCases {
        . "$file"
        ($inputValue -eq 1) | Should -eq $expected
    }
}

Which sources 30 files, so it's basically exponentially increasing the amount of setup. Any quick fixes for this currently?

[nohwnd]

@efie45 you are right this sucks, hopefully I will soon finish the parametric Describe and Context and then you should be able to provide the data directly to them instead of using foreach. I then need to look at how the name is expanded and expand it in the context of the block / test, and allow dot-notation so this would be possible.

How does this look to you?

$files = 'file1.ps1', 'file2.ps1', 'file3.ps1' 
$scenarios = @(
    @{ inputValue = 1; expected = $true }, @{ inputValue = 3; expected = $false }
    @{ inputValue = 4; expected = $false }, @{ inputValue = 5; expected = $false }
)

# -ForEach parameter would be added to Describe and Context
# and also to It where it would be aliased to TestCases. 
# It would not only array of hastables but any array, unlike the current -TestCase with this behavior: 
#  Each item is assigned to $_ variable, if the input array is an array of IDictionary (e.g. array of hastables)
#  it would additionally define each key as a variable (as it happens with TestCases in It right now)
Describe "Fake Tests" -ForEach $files {

    BeforeAll {
        $file = $_

        # source file once for all these tests
        . $file
    }

    It "<file> : <inputValue> is an integer" -TestCases $scenarios {
        $inputValue | Should -BeOfType [int]
    }

    It "<file> : <inputValue> should return <expected>" -TestCases $scenarios {
        ($inputValue -eq 1) | Should -eq $expected
    }
}

[nohwnd]

Added #1679 to link all the pieces together, please comment.

[efie45]

@nohwnd that looks great, thanks!

[arcanecode]

I just wanted to add my two cents to the thread. When I initially started with Pester, I wrote code much like stephanevg did in the original post. One of the things I loved about Pester was the ability to mix PowerShell code with the Pester tests. Made it very easy to do things like loop over each function validating things like does it have a help block, is it valid PowerShell code, do tests exist for it, etc.

I understand the (very) breaking changes necessitated by the new structure. However, it does break a good deal of my existing code. Right now I'm having to rewrite, and embed foreach loops inside each IT block to loop over my list of files, functions, etc.

The proposed use of a -ForEach parameter in the Describe would address these issues, as such I see this as a high priority need for the next version of Pester.

I've been away from PowerShell for a bit, so I admit to being shocked at the level of breaking changes that were caused by the new version 5.0.x version. Again I understand it, but it was still a surprise.

I'm getting back into it though as I am working on a new course for Pluralsight, and am including Pester tests as part of the course. I'm hopeful Pluralsight will let me create a new course in the near future for the new version 5, especially as my previous two courses are now outdated.

By the way I did two of the video courses on your Video section of the Pester.Dev site, under Robert Cain. If anyone wants to view and doesn't have a Pluralsight subscription contact me, I can give you a code for 30 free days there. My contact info is in my profile or you can go to my about me page (arcanecode.me).

Keep in mind these were done with older versions of both PowerShell and Pester. My new course will use PowerShell 7 on Windows, Linux and macOS.

[nohwnd]

I'm hopeful Pluralsight will let me create a new course in the near future for the new version 5, especially as my previous two courses are now outdated.

Are they looking for such course? I tried asking them about it few times, but no response.

[arcanecode]

I'm hopeful Pluralsight will let me create a new course in the near future for the new version 5, especially as my previous two courses are now outdated.

Are they looking for such course? I tried asking them about it few times, but no response.

We tentatively talked about it as a next course, after I finish my current course. But of course they don't give commitments until it's time to go forward, which is understandable.

In the course I'm currently developing I will be giving a brief Pester demo, but it won't be the main focus of the course. I did want to show "hey, you can write unit tests for your Powershell scripts just like you do for your C# (or JavaScript or whatever) projects.

The module I'm creating for the main demo addresses a common developer need, fake data. My DataFabricator will randomly generate fake, but realistic looking names, addresses, company names, products, and so on. I'm planning on releasing it as an open source project on github, so I've got help documentation (using PlatyPS right now to gen MD files from Help comments) and also want to include a suite of Pester tests.

At this point, the tests are the only thing I have left to code, so trying to decide whether to use 5.0.x Pester, or use the 5.1 beta. My course should go live sometime in Nov, so I want to have code as up to date as possible, so any guidance on this would be appreciated.

[nohwnd]

Use 5.1.0-beta, I am pretty sure it will be released till then.

[admin-simeon]

This -Foreach on Context is working great for me! How can I add the ForEach data to the Context/It Name in the Pester output?

[efie45]

@admin-simeon

You can use whatever kind of object you pass in by using <_> which references the current object in the foreach, similar to how $_ would.

$testCases = @('one', 'two')

Describe "Describe level: <_>" -Foreach $testCases {
    Context "Context level: <_>" {
        It "It level: <_>" {
            $true | Should -eq $true
        }
    }
}

Output:

 Context Context level: one
   [+] It level: one 116ms (76ms|40ms)
 Context Context level: two
   [+] It level: two 7ms (4ms|3ms)

Note: As you can see in this example there is currently a bug where some of the levels of the output aren't printed. Here there is no describe but the context shows up. I think they are fixing that in the current beta version.

There are a couple ways to reference these objects which are mentioned in the beta documentation here under the 'beta1' release. Hope that helps.

[admin-simeon]

@efie45 Thanks!

What about if I want to do something like this where my context is a combination of possible scenarios?

function Get-Combinations {
    [OutputType([hashtable])]
    [CmdletBinding()]
    param (
        [hashtable]$Source
    )
    $key = $Source.Keys | Select -First 1
    $nextSource = $Source.Clone()
    $nextSource.Remove($key)

    foreach ($value in @($Source.$key)) {

        if ($nextSource.Keys.Count -gt 0) {

            foreach ($next in Get-Combinations $nextSource) {
                $next.$key = $value
                Write-Output $next
            }
        }
        else {
            Write-Output @{ $key = $value }
        }
    }
}

 $booleanValues = @($true, $false)
    $contextValues = @{
        Export = $booleanValues
        Preview = $booleanValues
        Deploy = $booleanValues
        Comment = @('test comment')
        UpdateBaseline = $booleanValues
        NoBaseline = $booleanValues
        TenantPortalHasUpdates = $booleanValues
        TenantRepositoryHasUpdates = $booleanValues
        BaselineRepositoryHasUpdates = $booleanValues
        HasPriorSyncTags = $booleanValues
    }

Context "When running with context <[pscustomobject]_ | Out-String>" -Foreach (Get-Combinations $contextValues) {

[efie45]

I've never done anything like that so I would only be guessing. You'll probably have to try it out and see if you can get something like that to work. I'm not sure if you're able to use a script block inside the naming tag.

[wsmelton]

@admin-simeon played with this functionality this morning and was finally able to get something working.

I needed to be able to iterate over files in my project and verify the encoding. So I have a function within the test to check for that:

$moduleRoot = (Resolve-Path "$PSScriptRoot\..").Path
$allFiles = Get-ChildItem -Path $moduleRoot -Recurse -Filter "*.ps1" | Where-Object FullName -NotLike "$moduleRoot\tests\*"
Describe "Verifying module PS1 files" -Foreach $allFiles {
    BeforeAll {
        $name = $_.Name
        $fullName = $_.FullName
        $file = $_

        function Get-FileEncoding {
            [CmdletBinding()]
            Param (
                [Parameter(Mandatory = $True, ValueFromPipelineByPropertyName = $True)]
                [string]$Path
            )

            [byte[]]$byte = Get-Content -Encoding byte -ReadCount 4 -TotalCount 4 -Path $Path
            #Write-Host Bytes: $byte[0] $byte[1] $byte[2] $byte[3]

            # EF BB BF (UTF8)
            if ( $byte[0] -eq 0xef -and $byte[1] -eq 0xbb -and $byte[2] -eq 0xbf )
            { Write-Output 'UTF8' }

            # FE FF  (UTF-16 Big-Endian)
            elseif ($byte[0] -eq 0xfe -and $byte[1] -eq 0xff)
            { Write-Output 'Unicode UTF-16 Big-Endian' }

            # FF FE  (UTF-16 Little-Endian)
            elseif ($byte[0] -eq 0xff -and $byte[1] -eq 0xfe)
            { Write-Output 'Unicode UTF-16 Little-Endian' }

            # 00 00 FE FF (UTF32 Big-Endian)
            elseif ($byte[0] -eq 0 -and $byte[1] -eq 0 -and $byte[2] -eq 0xfe -and $byte[3] -eq 0xff)
            { Write-Output 'UTF32 Big-Endian' }

            # FE FF 00 00 (UTF32 Little-Endian)
            elseif ($byte[0] -eq 0xfe -and $byte[1] -eq 0xff -and $byte[2] -eq 0 -and $byte[3] -eq 0)
            { Write-Output 'UTF32 Little-Endian' }

            # 2B 2F 76 (38 | 38 | 2B | 2F)
            elseif ($byte[0] -eq 0x2b -and $byte[1] -eq 0x2f -and $byte[2] -eq 0x76 -and ($byte[3] -eq 0x38 -or $byte[3] -eq 0x39 -or $byte[3] -eq 0x2b -or $byte[3] -eq 0x2f) )
            { Write-Output 'UTF7' }

            # F7 64 4C (UTF-1)
            elseif ( $byte[0] -eq 0xf7 -and $byte[1] -eq 0x64 -and $byte[2] -eq 0x4c )
            { Write-Output 'UTF-1' }

            # DD 73 66 73 (UTF-EBCDIC)
            elseif ($byte[0] -eq 0xdd -and $byte[1] -eq 0x73 -and $byte[2] -eq 0x66 -and $byte[3] -eq 0x73)
            { Write-Output 'UTF-EBCDIC' }

            # 0E FE FF (SCSU)
            elseif ( $byte[0] -eq 0x0e -and $byte[1] -eq 0xfe -and $byte[2] -eq 0xff )
            { Write-Output 'SCSU' }

            # FB EE 28  (BOCU-1)
            elseif ( $byte[0] -eq 0xfb -and $byte[1] -eq 0xee -and $byte[2] -eq 0x28 )
            { Write-Output 'BOCU-1' }

            # 84 31 95 33 (GB-18030)
            elseif ($byte[0] -eq 0x84 -and $byte[1] -eq 0x31 -and $byte[2] -eq 0x95 -and $byte[3] -eq 0x33)
            { Write-Output 'GB-18030' }

            else
            { Write-Output 'ASCII' }
        }
    }
    Context "Validating <_>" {
        It "[$name] Should have UTF8 encoding" {
            Get-FileEncoding -Path $fullName | Should -Be 'UTF8'
        }

        It "[$name] Should have no trailing space" {
            ($file | Select-String "\s$" | Where-Object { $_.Line.Trim().Length -gt 0 } | Measure-Object).Count | Should -BeExactly 0
        }

        It "[$name] Should have no syntax errors" {
            $parseErrors | Should -BeNullOrEmpty
        }
    }
}

Then my output from the test (second run I added a file that was not UTF8 intentionally to make sure it failed properly:

[+] C:\git\thycotic.secretserver\tests\module.FileIntegrity.tests.ps1 8.6s (613ms|7.54s)
Tests completed in 8.66s
Tests Passed: 39, Failed: 0, Skipped: 1 NotRun: 0
[PS 5.1] [09:03:55] [  9.3 s ] C:\git\thycotic.secretserver\tests
 [2] > invoke-pester .\module.FileIntegrity.tests.ps1

Starting discovery in 1 files.
Discovery finished in 181ms.
[-] Verifying module PS1 files.Validating New-NotBom.ps1.[] Should have UTF8 encoding 89ms (85ms|4ms)
 Expected strings to be the same, but they were different.
 Expected length: 4
 Actual length:   28
 Strings differ at index 1.
 Expected: 'UTF8'
 But was:  'Unicode UTF-16 Little-Endian'
 at Get-FileEncoding -Path $fullName | Should -Be 'UTF8', C:\git\thycotic.secretserver\tests\module.FileIntegrity.tests.ps1:69
 at <ScriptBlock>, C:\git\thycotic.secretserver\tests\module.FileIntegrity.tests.ps1:69
Tests completed in 11.24s
Tests Passed: 41, Failed: 1, Skipped: 1 NotRun: 0

Full Gist of the test: https://gist.github.com/wsmelton/24bad5e8fbe4ca8f47b9bdb7f25e556d

[nohwnd]

This should be solved by the new -ForEach on Context, Describe and It. Closing.

[admin-simeon]

Seems like something broke in beta3 - it was printing correctly, but no longer...

  Context "When running with context <_>" -Foreach (Get-Combinations $contextValues | % { [pscustomobject]$_ }) {

now prints

Context When running with context <_>

in the test output

[madrum]

If it could help anyone finding this discussion, I put together a working example of using Pester to run PSScriptAnalyzer and generate NUnit XML, which can be used to published those test results in an ADO pipeline. The example can be found in this repo: https://github.com/DevOpsDrum/PSScriptAnalyzer-Pester