Open o-o00o-o opened 3 years ago
This is due to filters being processed in Discovery-phase, while the fullname (path of block/test) has variables expanded during the Run-phase.
@nohwnd Any thoughts on how to solve this? Moving the variable expansion to discovery would break support for using variables from BeforeAll/-Each in test/block names. We could expand name/path twice, replacing only known variables from Data during Discovery (New-Block/Test
) with a regex MatchEvaluator. However the filter might feel inconsistent as only some variables would be expanded.
Todo: The filter doesn't use ExpandedPath
atm. Could probably replace .Path -join '.'
with .ExpandedPath
in multiple places.
Thanks for the response @fflaten
I'm currently hacking this with the following at the top of my It block, but it would be good to get this resolved by the framework itself
if ($____Pester.Configuration.Filter.FullName.Value -and $____Pester.CurrentTest.ExpandedPath -notlike $____Pester.Configuration.Filter.FullName.Value) {
# work around the bug with pester
Set-ItResult -Skipped -Because "We skipped the test due to filters on FullName. ($($____Pester.CurrentTest.ExpandedPath)) -notlike ($($____Pester.Configuration.Filter.FullName.Value))"
break
}
However I found I couldn't use the same code in the BeforeEach. I didn't look into why that was too deeply - I just had to move the BeforeEach code out into the It block. At first glance it looked like the CurrentTest.ExpandedPath wasn't expanded in the BeforeEach either (perhaps a similar issue as you already identified)
A seperate issue I know but it would be good if we could also fix the purple text output when providing a FullName string/array. It always shows "Filter 'FullName' set to ('System.String[]')." and doesn't show the actual values. Tag works ok as you can see below
Starting discovery in 1 files.
Filter 'FullName' set to ('System.String[]').
Filter 'Tag' set to ('LOAD_FROM').
A seperate issue I know but it would be good if we could also fix the purple text output when providing a FullName string/array. It always shows "Filter 'FullName' set to ('System.String[]')." and doesn't show the actual values. Tag works ok as you can see below
Starting discovery in 1 files. Filter 'FullName' set to ('System.String[]'). Filter 'Tag' set to ('LOAD_FROM').
Moved to separate issue. 🙂
@fflaten Yes that is exactly why it does not work. Filtering is and mostly likely always will be Discovery only concept. This is a tax for being able to expand variables from the current scope into the name.
I am also considering making the -ForEach have a lazy version so you can evaluate the testcases during Run and not during Discovery, this would allow using BeforeAll variables in testcases (but not BeforeEach), and it would further complicate the way tests are run and filtered... So balancing all of this is difficult.
I still would like to have a -Filter where you can provide a scriptblock predicate and have it filter on whatever, but that would have to be intelligent about this and evaluated during discovery as well as during Run, but not in both.
In short I don't see any simple solution for this at the moment, unless ExpandedFullName
filter is added. It seems that filtering will be best kept as Discovery only concept and skipping will be used to skip the current test during run.
As @o-o00o-o shown above maybe Set-ItResult could get a -When parameter that takes a predicate and filters on the current object.
The hacky code seems to not work for BeforeAll because .CurrentBlock should be used instead of current test, and the test also does not re-check $skip on itself of the parent block when set this late.
But overall it seems doable.
Moving this to next milestone because it requires a new cmdlet to be published that would be tied to the internal state of Pester, and this requires thinking about how to best present that data, we probably should wrap the internal object into another object to protect it from being changed in the future. And discuss the shape of that new object.
General summary of the issue
Often I want to run a subset of my data driven tests - however it seems that I can't use -FullNameFilter for this as it doesn't apply to the post-processed name after expanding the hashtable values in <>'s
Describe your environment
Pester version : 5.1.1 C:...\Documents\WindowsPowerShell\Modules\pester\5.1.1\Pester.psm1 PowerShell version : 5.1.19041.610 OS version : Microsoft Windows NT 10.0.19041.0
Steps to reproduce
testing.ps1
now run
however if you run
Looking into the details of the not run you can see that the ExpandedName hasn't been expanded
Note that this same issue happens on describe and context blocks also. Nothing is expanded
Expected Behavior
I expect that test names/context/description etc are filtered after expansion
Current Behavior
no way to filter to a subset of data driven tests (!)