pester / Pester

Pester is the ubiquitous test and mock framework for PowerShell.
https://pester.dev/
Other
3.11k stars 473 forks source link

Invoking Pester configuration with multiple test scripts containing differing parameters throws error on discovery #2299

Closed jazuntee closed 1 year ago

jazuntee commented 1 year ago

Checklist

What is the issue?

When invoking Pester configuration with multiple test scripts, adding a script parameter to one of the test scripts (that does not exist on the others) throws the following error during discovery on all other test scripts loaded AFTER the script containing that new parameter.

Pester v5.4.0

Starting discovery in 9 files.
[-] Discovery in C:\Users\jasoth\Source\Repos\MSIdentityTools\tests\Revoke-MsIdServicePrincipalConsent.tests.ps1 failed with:
System.Management.Automation.ParameterBindingException: A parameter cannot be found that matches parameter name 'ModulePrefix'.
   at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
   at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)
   at System.Management.Automation.PSScriptCmdlet.RunClause(Action`1 clause, Object dollarUnderbar, Object inputToProcess)
   at System.Management.Automation.CommandProcessorBase.Complete()
at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2995
at Invoke-File, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 3004
at Invoke-BlockContainer, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2917
at Discover-Test, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 1464
at Invoke-Test, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2437
at Invoke-Pester<End>, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 5272
at <ScriptBlock>, C:\Users\jasoth\Source\Repos\MSIdentityTools\build\Test-PSModule.ps1: line 46
at <ScriptBlock>, <No file>: line 1
[-] Discovery in C:\Users\jasoth\Source\Repos\MSIdentityTools\tests\Test-MgCommandPrerequisites.tests.ps1 failed with:
System.Management.Automation.ParameterBindingException: A parameter cannot be found that matches parameter name 'ModulePrefix'.
   at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
   at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0)
   at System.Management.Automation.PSScriptCmdlet.RunClause(Action`1 clause, Object dollarUnderbar, Object inputToProcess)
   at System.Management.Automation.CommandProcessorBase.Complete()
at <ScriptBlock>, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2995
at Invoke-File, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 3004
at Invoke-BlockContainer, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2917
at Discover-Test, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 1464
at Invoke-Test, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 2437
at Invoke-Pester<End>, C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1: line 5272
at <ScriptBlock>, C:\Users\jasoth\Source\Repos\MSIdentityTools\build\Test-PSModule.ps1: line 46
at <ScriptBlock>, <No file>: line 1
Discovery found 508 tests in 3.04s.

Expected Behavior

I expected all other test scripts to be unaffected by the new parameter in one of the test scripts.

Invoking Pester Configuration with Data parameter containing value for the named parameter, would be mapped to the one script containing that parameter and ignored for all test scripts that did not contain that script parameter.

Likewise, Invoking Pester Configuration with Data parameter NOT containing a value for script parameter would result in default script parameter value in the test script containing that parameter and no issue with scripts that do not.

Steps To Reproduce

TestScript1.ps1

[CmdletBinding()]
param (
    [Parameter(Mandatory = $false)]
    [string] $ModulePath = ".\src\*.psd1"
)

TestScript2.ps1 (Contains Unique ModulePrefix Parameter)

[CmdletBinding()]
param (
    [Parameter(Mandatory = $false)]
    [string] $ModulePath = ".\src\*.psd1",
    [Parameter(Mandatory = $false)]
    [string] $ModulePrefix = "Testing"
)

TestScript3.ps1

[CmdletBinding()]
param (
    [Parameter(Mandatory = $false)]
    [string] $ModulePath = ".\src\*.psd1"
)

TestScript4.ps1

[CmdletBinding()]
param (
    [Parameter(Mandatory = $false)]
    [string] $ModulePath = ".\src\*.psd1"
)

Invoke Pester Configuration

$PesterConfiguration = New-PesterConfiguration (Import-PowerShellDataFile $PesterConfigurationFileInfo.FullName)
$PesterConfiguration.Run.Container = New-PesterContainer -Path $ModuleTestsDirectoryInfo.FullName -Data @{ ModulePath = $ModuleManifestFileInfo.FullName }
$PesterRun = Invoke-Pester -Configuration $PesterConfiguration

Discovery on TestScript3.ps1 and TestScript4.ps1 throws error because they followed TestScript2.ps1 which contained a parameter that does not exist on TestScript3.ps1 and TestScript4.ps1.

Describe your environment

Pester version : 5.4.0 C:\Program Files\WindowsPowerShell\Modules\Pester\5.4.0\Pester.psm1 PowerShell version : 7.3.2 OS version : Microsoft Windows NT 10.0.22621.0

Possible Solution?

No response

fflaten commented 1 year ago

Thanks for the report. This is the same issue as #2073.

A workaround for now is to call New-PesterContainer once per path with it's own copy of the hash table.

$containers = $ModuleTestsDirectoryInfo | % { New-PesterContainer -Path $_.FullName -Data @{ ModulePath = $ModuleManifestFileInfo.FullName } }
$PesterConfiguration.Run.Container = $containers