If a non existing exclude tag is given amonst existing exclude tag, it still runs the tests tagged with the given excludetag.

ChefkeGremmen commented 2 months ago

Checklist

[X] Issue has a meaningful title
[X] I have searched the existing issues. See all issues
[X] I have tested using the latest version of Pester. See Installation and update guide.

What is the issue?

Given that I have a few tests, some with tags like 'Integrationtest'. If I now create a config like;

$config = New-PesterConfiguration
$config.Filter.ExcludeTag = @('Integrationtest','Windowsonly')

Where there is not a single Describe, Context or It tagged with 'WindowsOnly' (in other words, 'WindowsOnly' does not exist) If I then run Pester, the tests tagged with 'IntegrationTest' are still run. Not a single error message (also not in Diagnostic level verbosity) is given on 'WindowsOnly'.

Expected Behavior

I would expect the tests covered by the 'Integrationtest' tag to be excluded. Also a warning that the given tag was not discovered would be nice.

Steps To Reproduce

See above.

Describe your environment

Pester version : 5.6.1 PowerShell version : 7.4.5 OS version : Microsoft Windows NT 10.0.22621.0

Possible Solution?

No response

fflaten commented 2 months ago

I'm unable to reproduce as you can see below. Could you provide a complete repro?

$config = New-PesterConfiguration
$config.Run.ScriptBlock = {
    Describe 'd' {
        It 'i' -Tag 'Integrationtest' {
            1 | Should -Be 1
        }
        It 'w' {
            1 | Should -Be 1
        }
    }
}
$config.Filter.ExcludeTag = @('Integrationtest','Windowsonly')
$config.Output.Verbosity = 'Detailed'
Invoke-Pester -Configuration $config

# Output

Pester v5.6.1

Starting discovery in 1 files.
Discovery found 2 tests in 160ms.
Filter 'ExcludeTag' set to ('Integrationtest', 'Windowsonly').
Filters selected 1 tests to run.
Running tests.
Describing d
  [+] w 137ms (105ms|31ms)
Tests completed in 896ms
Tests Passed: 1, Failed: 0, Skipped: 0, Inconclusive: 0, NotRun: 1

ChefkeGremmen commented 2 months ago

I am so sorry to have wasted your time ... 😥 I was able to reproduce it, and it does not seem to be an issue in Pester, but rather the way pester was invoked. First of all, I saw it happen with a Tag on a describe, so that is what I created first. Further, we pass the exclude tags as a commandline parameter:

[CmdletBinding()]
param (
    [string[]] $excludeTags = @())

$config = New-PesterConfiguration
$config.Run.ScriptBlock = {
    Describe 'd' -Tag 'Integrationtest' {
        It 'i' {
            1 | Should -Be 1
        }
        It 'w' {
            1 | Should -Be 1
        }
    }

    Describe 'iets' {
        It 'heel' {
            1 | Should -Be 1
        }
        It 'anders' {
            1 | Should -Be 1
        }
    }
}
$config.Filter.ExcludeTag = $excludeTags
$config.Output.Verbosity = 'Detailed'
Invoke-Pester -Configuration $config

When now invoking this script from the commandline with;

pwsh .\example.Tests.ps1 -excludeTags Integrationtest,Windowsonly

You will notice the following line:

Filter 'ExcludeTag' set to ('Integrationtest,Windowsonly').

It interpreted the Integrationtest,Windowsonly part as one big string.

Anyway, thanks for you time and keep up the good work with Pester.

pester / Pester