pester / Pester

Pester is the ubiquitous test and mock framework for PowerShell.
https://pester.dev/
Other
3.11k stars 473 forks source link

[v5] ExcludePath argument and configuration item don't work for directories, only files #1575

Open JustinGrote opened 4 years ago

JustinGrote commented 4 years ago

1. General summary of the issue

ExcludePath only works for files, not directories, even though the description says it should work for directories. image

2. Describe Your Environment

Pester version : 5.0.0 C:\Users\jgrote\Documents\PowerShell\Modules\Pester\5.0.0\Pester.psd1 PowerShell version : 7.0.1 OS version : Microsoft Windows NT 10.0.19041.0

nohwnd commented 4 years ago

Could you run it with diagnostic output? So I can see if there is anything usefule, or if more messages need to be added. I don't think any logging is done for this part of the process (yet).

JustinGrote commented 4 years ago
~\\Sandbox\Pesterinvoke-pester -ExcludePath C:\Users\jgrote\Projects\Sandbox\Pester\ExcludeMe -Output Diagnostic

Discovery: Starting test discovery in 2 test containers. 

Starting discovery in 2 files.
Discovery: Discovering tests in C:\Users\jgrote\Projects\Sandbox\Pester\ExcludeMe\ExcludeMe.Tests.ps1 
Discovering in C:\Users\jgrote\Projects\Sandbox\Pester\ExcludeMe\ExcludeMe.Tests.ps1.
Found 1 tests. 96ms
Discovery: Found 1 tests in 126 ms 
Discovery: Discovering tests in C:\Users\jgrote\Projects\Sandbox\Pester\Tests\Describe.Tests.ps1 
Discovering in C:\Users\jgrote\Projects\Sandbox\Pester\Tests\Describe.Tests.ps1.
Found 1 tests. 8ms
Discovery: Found 1 tests in 36 ms 
Discovery: Processing discovery result objects, to set root, parents, filters etc. 
Filter: (Exclude) Block will be included in the run, because there were no include filters, and will let its children to determine whether or not it should run. 
Filter: (Exclude.Should Not Run) Test will be included in the run, because there were no include filters so all tests are included unless they match exclude rule.        
Filter: (d) Block will be included in the run, because there were no include filters, and will let its children to determine whether or not it should run. 
Filter: (d.shouldrun) Test will be included in the run, because there were no include filters so all tests are included unless they match exclude rule. 
Discovery finished in 302ms.
Discovery: Test discovery finished. 

Running tests from 'C:\Users\jgrote\Projects\Sandbox\Pester\ExcludeMe\ExcludeMe.Tests.ps1'
Describing Exclude
I WAS NOT EXCLUDED!
  [+] Should Not Run 17ms (13ms|4ms)

Running tests from 'C:\Users\jgrote\Projects\Sandbox\Pester\Tests\Describe.Tests.ps1'
Describing d
I Should Run!
  [+] shouldrun 26ms (23ms|3ms)
Tests completed in 745ms
Tests Passed: 2, Failed: 0, Skipped: 0 NotRun: 0
nohwnd commented 4 years ago

It uses -like to compare, so you would be able to exclude the directory using <dir>/*, I remember doing it that way because I can't really be sure if the path is directory or a file, when it itself uses a wildcard.

I could add an implicit, <dirseparator>* filter to all the entries that don't end with it to get:

c:\temp\file1.tests.ps1
c:\temp
c:\temp\*\*\*.acceptance.tests*

VVVV

c:\temp\file1.tests.ps1
c:\temp\file1.tests.ps1\*
c:\temp
c:\temp\*
c:\temp\*\*\*.acceptance.tests*
c:\temp\*\*\*.acceptance.tests*\*

This would work for both a file named temp in the C:\ drive, and a directory named file1.tests.ps1 in a temp folder, and a wildcarded path that can match anything.

But looking at it, it's nice to have but has a lot of edge cases, so better documentation is probably the way to go.

JustinGrote commented 4 years ago

I like this thinking, I don't see the edge cases as long as you check if it's a directory first. Even with a lot of files this is very efficient and cross platform:

[String[]]$PathsToMatch = foreach ($PathFilterItem in $PathFilter) {
  if ([io.directory]::Exists($PathFilterItem)) {
    Join-Path $PathFilterItem '*'
  } else {
    $PathFilterItem
  }
}

I think a user would want to intuitively specify a directory (as I did) without appending the wildcard so this is IMHO the best way to do it and still allow partial wildcard matches for files.

You could also use [io.path]::Combine instead of Join-Path to maybe shave off a millisecond or two :)

asears commented 4 years ago

A .pesterignore file seems to be another option, with wildcard parameters (similar to .gitignore), and specific path (-ExcludePath) or wildcard parameters in the command line.

First impression would be that argument -ExcludePath "excludes a path" and not a "path Like" by default. Perhaps that would be -ExcludePaths perhaps. or -ExcludePathsLike

Currently have similar issue with Code Coverage and specifying paths / excluding paths in a dynamic fashion, for testing & code coverage. Sometimes we just rename ps1 to ps1.ignore to exclude specific files. Good to know it accepts wildcards for our case.

JustinGrote commented 4 years ago

I dont think we need yet another dot file, especially because you could do this yourself with Get-Content and don't require pester to do it for you.