Should -HaveParameter yields a RunTime exception when used on a mocked function

johndog commented 4 years ago

1. General summary of the issue

Title sums it up.

To reproduce, use Invoke-Pester on the following tests.ps1 code:

    function TestFunction($Parameter1) {
    }

    Mock TestFunction {}

    It 'should have Parameter1' {
        Get-Command TestFunction | Should -HaveParameter "Parameter1"
    }

RuntimeException Result:

    [-] should have Parameter1 7ms
      RuntimeException: You cannot call a method on a null-valued expression.
      at Should-HaveParameter, C:\Program Files\WindowsPowerShell\Modules\pester\4.9.0\Functions\Assertions\HaveParameter.ps1: line 142
      at <ScriptBlock>, C:\Users\johnd\OneDrive\scripts\StateChangingDemo.tests.ps1: line

2. Describe Your Environment

Pester version : 4.9.0 C:\Program Files\WindowsPowerShell\Modules\Pester\4.9.0\Pester.psd1 PowerShell version : 5.1.19041.1 OS version : Microsoft Windows NT 10.0.19041.0

3. Expected Behavior

Should should either reject the use of a mock explicitly, or treat the mock as if it was the original function and pass the test. A RunTime exception shouldn't leak through.

5. Fix consideration

The exception occurs in this line in HaveParameter.ps1 (line 142):

$hasKey = $ActualValue.Parameters.PSBase.ContainsKey($ParameterName)

$ActualValue.Parameters at this point is $null, so the access to PSBase fails.

The check for whether the function is mocked would need to take place earlier in the function, either to reject the use of Should, or to look up the underlying command and act on that.

6. Context

I was trying to write tests for a function. It wasn't necessary to test parameters on a mocked version of the function, but the nature of the error makes it hard to figure out why it doesn't work.

nohwnd commented 4 years ago

Thanks for reporting, will have a look.

nohwnd commented 4 years ago

Cannot reproduce this in 4.10.1 nor 5.0.2 (when putting the function and mock into BeforeAll). If you can reproduce on the latest version please provide full repro.

johndog commented 4 years ago

Still reproduces for me, on two different computers:

PS C:\Users\johnd> invoke-pester .\OneDrive\repos\pshlib\bugs\parm.tests.ps1
Pester v4.10.1
Executing all tests in '.\OneDrive\repos\pshlib\bugs\parm.tests.ps1'

Executing script .\OneDrive\repos\pshlib\bugs\parm.tests.ps1
  [-] should have Parameter1 178ms
    RuntimeException: You cannot call a method on a null-valued expression.
    at Should-HaveParameter, C:\Program Files\WindowsPowerShell\Modules\Pester\4.10.1\Functions\Assertions\HaveParameter.ps1: line 142
    at <ScriptBlock>, C:\Users\johnd\OneDrive\repos\pshlib\bugs\parm.tests.ps1: line 7
Tests completed in 1.01s
Tests Passed: 0, Failed: 1, Skipped: 0, Pending: 0, Inconclusive: 0

parm.tests.ps1 is just a literal paste of the repro code.

Both of these machines are running Windows Insider builds... I actually don't have anything else. Do you have a machine with an insider build?

PS C:\Users\johndog\OneDrive\repos\pshlib\bugs> $PSVersionTable

Name                           Value                                                                                                                                                                               
----                           -----                                                                                                                                                                               
PSVersion                      5.1.20180.1000                                                                                                                                                                      
PSEdition                      Desktop                                                                                                                                                                             
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                             
BuildVersion                   10.0.20180.1000                                                                                                                                                                     
CLRVersion                     4.0.30319.42000                                                                                                                                                                     
WSManStackVersion              3.0                                                                                                                                                                                 
PSRemotingProtocolVersion      2.3                                                                                                                                                                                 
SerializationVersion           1.1.0.1

fflaten commented 2 years ago

Still able to repro in 5.3.3. Get-Command is lazy-loading the parameters. Can repro with Set-Alias, but not built-in aliases as they've probably been called sometime. So CommandInfo is passed to Should without Parameters being ready. Simply accessing it before calling Should solves it. Ex.

Describe 'd1' {
    BeforeAll {
        function TestFunction($Parameter1) { }
        Mock TestFunction {}
    }
    It 'should have Parameter1' {
        Get-Command TestFunction | Where-Object Parameters | Should -HaveParameter 'Parameter1'
    }
}

Similar to the PSv2-fix here, but it doesn't solve this one: https://github.com/pester/Pester/blob/dd7d09eb730c8d9fc7ac746aeecf750bf98b9d00/src/functions/assertions/HaveParameter.ps1#L192-L193 The original commit doesn't say much about it.

fflaten commented 2 years ago

It looks like PowerShell tries to lazy-load the metadata for the original function when getting it's parameters. When doing that, PowerShell doesn't look in the local scope in the caller session state (testfile) were the function was created, so it can't find it. If TestFunction was from a module, it would work - even with local alias.

PowerShell-native repro:

Get-Module funcModule | Remove-Module
New-Module funcModule { 
    function moduleFunc ($Param1234) { }
} | Import-Module

Get-Module testModule | Remove-Module
New-Module testModule { 
    function test($localAlias, $localFunc, $moduleFunc, $localAlias2) {
        "Running in $($ExecutionContext.SessionState.Module.name)"
        "Script Function $($localFunc.Name) contains parameters $($null -ne $localFunc.Parameters)"
        "Script Alias $($localAlias.Name) contains parameters $($null -ne $localAlias.Parameters)"
        "Module Function $($moduleFunc.Name) contains parameters $($null -ne $moduleFunc.Parameters)"
        "Script Alias $($localAlias2.Name) for module func contains parameters $($null -ne $localAlias2.Parameters)"

        #Try accessing AliasInfo.Parameters for the local function
        $localAlias.Parameters.PSBase.ContainsKey('MyParam1')
    }
} | Import-Module

# running in a local scope. dot-sourcing this would work because the function is created in the root
& {
    function scriptFunc ($MyParam1) { }
    Set-Alias -Name alias1 -Value scriptFunc
    Set-Alias -Name alias2 -Value moduleFunc
    test -localFunc (Get-Command scriptFunc) -localAlias (Get-Command alias1) -moduleFunc (Get-Command moduleFunc) -localAlias2 (Get-Command alias2)
}

# Output
Running in testModule
Script Function scriptFunc contains parameters True
Script Alias alias1 contains parameters False
Module Function moduleFunc contains parameters True
Script Alias alias2 for module func contains parameters True
InvalidOperation:
Line |
  10 |          $localAlias.Parameters.PSBase.ContainsKey('MyParam1')
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

pester / Pester