pester / Pester

Pester is the ubiquitous test and mock framework for PowerShell.
https://pester.dev/
Other
3.11k stars 473 forks source link

It's probably me, but Pester is not executing my tests #1459

Closed DarkLite1 closed 3 years ago

DarkLite1 commented 4 years ago

1. General summary of the issue

There's this strange thing... I wrote some test code for a script. In the test the script is called correctly but the code after calling the script, the actual tests, are not executed. When the it clause is finished it's done in a green color stating Success.

According to my logic, when tests are not executed, or there is an error in the script, the Pester test should fail.

See attachment for the files. When you set a breakpoint on a line after the script call (ex. line 46) in the test file it will never be reached. Pester test.zip

2. Describe Your Environment

Pester version     : 4.10.1 C:\Program Files\WindowsPowerShell\Modules\Pester\4.10.1\Pester.psd1
PowerShell version : 5.1.14393.3471
OS version         : Microsoft Windows NT 10.0.14393.0

3. Expected Behavior

When the script fails or not all tests are ran, the test should fail not pass.

4.Current Behavior

The test passes.

DarkLite1 commented 4 years ago

Figured it out, in the function Get-FolderContentHC I used:

if ($scannedInheritedFolders.ContainsKey($Path)) {Continue}

instead of:

if ($scannedInheritedFolders.ContainsKey($Path)) {Return}

This made the script exit the Pester it clause and not execute the remaining tests. Still, I believe that Pester should check in the it clause to see if there are any tests. If then, by reasons unknown, these tests are not executed it should flag the it clause as failed

A mistake like this would otherwise never be caught and the author would simply assume that all is correct, Which is not the case.

nohwnd commented 4 years ago

Just today I wrote a toy test framework that starts the test as inconclusive, and only marks it as success if there is a passing assertion. Is that what you meant by "Pester should check in the it clause to see if there are any tests." In this context tests = assertions?

DarkLite1 commented 4 years ago

Indeed, that's what I mean. Pester should start with the status that a test, or assertion as you call it, is failed by default.

In the example case above the assertions where never reached for execution, so Pester assumed all tests passed while they actually haven't been executed at all.

nohwnd commented 3 years ago

Imho this is built in in Pester v5, test is failed when it should run but it is not executed.