search

TylerLeonhardt / vscode-pester-test-adapter

MIT License
33 stars 13 forks source link

Discover Fails 'Get-ScriptAnalyzerRule' is not recognized #42

Open obsidian33 opened 3 years ago

obsidian33 commented 3 years ago

I have a test that uses Get-ScriptAnalyzerRule which I think is preventing the GUI from showing my tests. When I hit the run button I get the error:

System.Management.Automation.CommandNotFoundException: The term 'Get-ScriptAnalyzerRule' is not recognized as the name of a cmdlet, function, script file, or operable program.

When I manually run this test with 'Invoke-Pester' there are no issues.

The test is pretty simple, here is the snippet:

Describe 'PSScriptAnalyzer Rules' -Tag 'Meta' {
    $rules = Get-ScriptAnalyzerRule | ForEach-Object {
        @{ RuleName = $_.RuleName }
    }

    BeforeAll {
        $analysis = Invoke-ScriptAnalyzer -Path $ProjectRoot -Recurse
    }

    It "Should pass <RuleName>" -ForEach $rules {
        if (($analysis) -and ($analysis.RuleName -contains $RuleName)) {
            $analysis | Where-Object RuleName -EQ $RuleName -OutVariable failures | Out-Default
            $failures.Count | Should -Be 0
        }
    }
}
TylerLeonhardt commented 3 years ago

In a PowerShell console outside of VS Code, is Get-ScriptAnalyzerRule available?

My suspicion is that you're relying on the PSScriptAnalyzer that ships in the PowerShell extension (and is available in the PowerShell Integrated Console) but the tests don't run in that, they get run in what's called a VS Code Task which is just a normal PowerShell console.

obsidian33 commented 3 years ago

@TylerLeonhardt you're right I thought I had it installed. After installing the module I no longer have the previous error but the GUI is still blank even though the VS Code Task runs smoothly.

Importing Pester module...

Starting discovery in 2 files.
Discovery finished in 769ms.
[+] Invoke-MNetSqlQuery.Tests.ps1 2.48s (1.53s|645ms)
[+] MerlinNet.Tests.ps1 7s (6.08s|502ms)
Tests completed in 9.53s
Tests Passed: 67, Failed: 0, Skipped: 0 NotRun: 0

Terminal will be reused by tasks, press any key to close it.

image

I realize this may be a separate and unrelated to what I originally posted. Should I close this issue and submit another for this?

TylerLeonhardt commented 3 years ago

In the Output pane under "Pester Explorer Log" in the drop down can you share what's in there?

obsidian33 commented 3 years ago 
[2021-05-03 21:00:54.633] [INFO] Test Explorer found
[2021-05-03 21:00:54.633] [INFO] Creating adapter for c:\SWDev\PowerShell\MerlinNet
[2021-05-03 21:00:54.633] [INFO] Initializing Pester adapter
[2021-05-03 21:00:54.633] [INFO] Initializing Pester test runner.
[2021-05-03 21:00:54.639] [INFO] Registering adapter for c:\SWDev\PowerShell\MerlinNet
[2021-05-03 21:00:54.640] [INFO] Loading Pester tests
[2021-05-03 21:00:54.640] [INFO] Initialization finished
[2021-05-03 21:00:55.158] [DEBUG] Found 2 paths
[2021-05-03 21:01:00.051] [DEBUG] Using Windows PowerShell (x64) at: C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe
[2021-05-03 21:01:00.051] [DEBUG] 
$Path = @(
    'c:\SWDev\PowerShell\MerlinNet\Tests\MerlinNet.Tests.ps1'
'c:\SWDev\PowerShell\MerlinNet\Tests\Invoke-MNetSqlQuery.Tests.ps1'
)

$VerbosePreference = 'Ignore'
$WarningPreference = 'Ignore'
$DebugPreference = 'Ignore'
Import-Module Pester -MinimumVersion 5.0.0 -ErrorAction Stop
function Discover-Test
{
    [CmdletBinding()]
    param(
        [Parameter(Mandatory)]
        [String[]] $Path,
        [String[]] $ExcludePath
    )
    & (Get-Module Pester) { 
        param (
            $Path, 
            $ExcludePath,
            $SessionState)

        Reset-TestSuiteState
        # to avoid Describe thinking that we run in interactive mode
        $invokedViaInvokePester = $true
        $files = Find-File -Path $Path -ExcludePath $ExcludePath -Extension $PesterPreference.Run.TestExtension.Value
        $containers = foreach ($f in $files) {
            <# HACK: We check to see if there is a single Describe block in the file so that we don't accidentally execute code that shouldn't need to be executed. #>
            if (!(Select-String -Path $f -SimpleMatch 'Describe')) {
                continue
            }
            New-BlockContainerObject -File (Get-Item $f)
        }
        Find-Test -BlockContainer $containers -SessionState $SessionState } -Path $Path -ExcludePath $ExcludePath -SessionState $PSCmdlet.SessionState
}

function New-SuiteObject ($Block) { 
    [PSCustomObject]@{
        type = 'suite'
        id = $Block.ScriptBlock.File + ';' + $Block.StartLine
        file = $Block.ScriptBlock.File
        line = $Block.StartLine - 1
        label = $Block.Name
        children = [Collections.Generic.List[Object]]@()
    }
}

function New-TestObject ($Test) { 
    [PSCustomObject]@{
        type = 'test'
        id = $Test.ScriptBlock.File + ';' + $Test.StartLine
        file = $Test.ScriptBlock.File
        line = $Test.StartLine - 1
        label = $Test.Name
    }
}

function fold ($children, $Block) {
    foreach ($b in $Block.Blocks) { 
        $o = (New-SuiteObject $b)
        $children.Add($o)
        fold $o.children $b
    }

    $hashset = [System.Collections.Generic.HashSet[string]]::new()
    foreach ($t in $Block.Tests) {
        $key = "$($t.ExpandedPath):$($t.StartLine)"
        if ($hashset.Contains($key)) {
            continue
        }
        $children.Add((New-TestObject $t))
        $hashset.Add($key) | Out-Null
    }
    $hashset.Clear() | Out-Null
}

$found = Discover-Test -Path $Path

# whole suite
$suite = [PSCustomObject]@{
    Blocks = [Collections.Generic.List[Object]] $found
    Tests = [Collections.Generic.List[Object]]@()
}

$testSuiteInfo = [PSCustomObject]@{
    type = 'suite'
    id = 'root'
    label = 'Pester'
    children = [Collections.Generic.List[Object]]@()
}

foreach ($file in $found) {
    $fileSuite = [PSCustomObject]@{
        type = 'suite'
        id = $file.BlockContainer.Item.FullName
        file = $file.BlockContainer.Item.FullName
        label = $file.BlockContainer.Item.Name
        children = [Collections.Generic.List[Object]]@()
    }
    $testSuiteInfo.children.Add($fileSuite)
    fold $fileSuite.children $file
}

$testSuiteInfo | ConvertTo-Json -Depth 100

[2021-05-03 21:01:00.745] [ERROR] stderr: Import-Module : The value Ignore is not supported for an ActionPreference variable. The provided value should be used 

[2021-05-03 21:01:00.745] [ERROR] stderr: only as a value for a preference parameter, and has been replaced by the default value. For more information, see the 
Help topic, "about_Preference_Variables."
At line:10 char:1
+ Import-Module Pester -MinimumVersion 5.0.0 -ErrorAction Stop

[2021-05-03 21:01:00.745] [ERROR] stderr: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Import-Module], NotSupportedException
    + FullyQualifiedErrorId : System.NotSupportedException,Microsoft.PowerShell.Commands.ImportModuleCommand

[2021-05-03 21:01:00.745] [ERROR] stderr:  

[2021-05-03 21:01:00.780] [ERROR] stderr: The expression after '&' in a pipeline element produced an object that was not valid. It must result in a command 

[2021-05-03 21:01:00.781] [ERROR] stderr: name, a script block, or a CommandInfo object.
At line:19 char:7
+     & (Get-Module Pester) {
+       ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

[2021-05-03 21:01:00.781] [ERROR] stderr:     + FullyQualifiedErrorId : BadExpression

[2021-05-03 21:01:00.781] [ERROR] stderr:  

[2021-05-03 21:01:00.960] [DEBUG] stdout: {
    "type":  "suite",
    "id":  "root",
    "label":  "Pester",
    "children":  [

                 ]
}

[2021-05-03 21:01:00.992] [DEBUG] child process exited with code 0
[2021-05-03 21:01:00.995] [ERROR] Unable to parse JSON data from Pester test discovery script. Contents: {
    "type":  "suite",
    "id":  "root",
    "label":  "Pester",
    "children":  [

                 ]
}

[2021-05-03 21:01:12.448] [INFO] Running Pester tests ["root"]
[2021-05-03 21:01:41.204] [DEBUG] Found 1 paths
[2021-05-03 21:01:41.204] [DEBUG] Using Windows PowerShell (x64) at: C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe
[2021-05-03 21:01:41.205] [DEBUG] 
$Path = @(
    'c:\SWDev\PowerShell\MerlinNet\Tests\Invoke-MNetSqlQuery.Tests.ps1'
)

$VerbosePreference = 'Ignore'
$WarningPreference = 'Ignore'
$DebugPreference = 'Ignore'
Import-Module Pester -MinimumVersion 5.0.0 -ErrorAction Stop
function Discover-Test
{
    [CmdletBinding()]
    param(
        [Parameter(Mandatory)]
        [String[]] $Path,
        [String[]] $ExcludePath
    )
    & (Get-Module Pester) { 
        param (
            $Path, 
            $ExcludePath,
            $SessionState)

        Reset-TestSuiteState
        # to avoid Describe thinking that we run in interactive mode
        $invokedViaInvokePester = $true
        $files = Find-File -Path $Path -ExcludePath $ExcludePath -Extension $PesterPreference.Run.TestExtension.Value
        $containers = foreach ($f in $files) {
            <# HACK: We check to see if there is a single Describe block in the file so that we don't accidentally execute code that shouldn't need to be executed. #>
            if (!(Select-String -Path $f -SimpleMatch 'Describe')) {
                continue
            }
            New-BlockContainerObject -File (Get-Item $f)
        }
        Find-Test -BlockContainer $containers -SessionState $SessionState } -Path $Path -ExcludePath $ExcludePath -SessionState $PSCmdlet.SessionState
}

function New-SuiteObject ($Block) { 
    [PSCustomObject]@{
        type = 'suite'
        id = $Block.ScriptBlock.File + ';' + $Block.StartLine
        file = $Block.ScriptBlock.File
        line = $Block.StartLine - 1
        label = $Block.Name
        children = [Collections.Generic.List[Object]]@()
    }
}

function New-TestObject ($Test) { 
    [PSCustomObject]@{
        type = 'test'
        id = $Test.ScriptBlock.File + ';' + $Test.StartLine
        file = $Test.ScriptBlock.File
        line = $Test.StartLine - 1
        label = $Test.Name
    }
}

function fold ($children, $Block) {
    foreach ($b in $Block.Blocks) { 
        $o = (New-SuiteObject $b)
        $children.Add($o)
        fold $o.children $b
    }

    $hashset = [System.Collections.Generic.HashSet[string]]::new()
    foreach ($t in $Block.Tests) {
        $key = "$($t.ExpandedPath):$($t.StartLine)"
        if ($hashset.Contains($key)) {
            continue
        }
        $children.Add((New-TestObject $t))
        $hashset.Add($key) | Out-Null
    }
    $hashset.Clear() | Out-Null
}

$found = Discover-Test -Path $Path

# whole suite
$suite = [PSCustomObject]@{
    Blocks = [Collections.Generic.List[Object]] $found
    Tests = [Collections.Generic.List[Object]]@()
}

$testSuiteInfo = [PSCustomObject]@{
    type = 'suite'
    id = 'root'
    label = 'Pester'
    children = [Collections.Generic.List[Object]]@()
}

foreach ($file in $found) {
    $fileSuite = [PSCustomObject]@{
        type = 'suite'
        id = $file.BlockContainer.Item.FullName
        file = $file.BlockContainer.Item.FullName
        label = $file.BlockContainer.Item.Name
        children = [Collections.Generic.List[Object]]@()
    }
    $testSuiteInfo.children.Add($fileSuite)
    fold $fileSuite.children $file
}

$testSuiteInfo | ConvertTo-Json -Depth 100

[2021-05-03 21:01:41.809] [ERROR] stderr: Import-Module : The value Ignore is not supported for an ActionPreference variable. The provided value should be used 

[2021-05-03 21:01:41.809] [ERROR] stderr: only as a value for a preference parameter, and has been replaced by the default value. For more information, see the 
Help topic, "about_Preference_Variables."
At line:9 char:1
+ Import-Module Pester -MinimumVersion 5.0.0 -ErrorAction Stop

[2021-05-03 21:01:41.809] [ERROR] stderr: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Import-Module], NotSupportedException
    + FullyQualifiedErrorId : System.NotSupportedException,Microsoft.PowerShell.Commands.ImportModuleCommand

[2021-05-03 21:01:41.809] [ERROR] stderr:  

[2021-05-03 21:01:41.835] [ERROR] stderr: The expression after '&' in a pipeline element produced an object that was not valid. It must result in a command 

[2021-05-03 21:01:41.835] [ERROR] stderr: name, a script block, or a CommandInfo object.
At line:18 char:7
+     & (Get-Module Pester) {

[2021-05-03 21:01:41.835] [ERROR] stderr: +       ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

[2021-05-03 21:01:41.835] [ERROR] stderr:     + FullyQualifiedErrorId : BadExpression

[2021-05-03 21:01:41.981] [DEBUG] stdout: {
    "type":  "suite",
    "id":  "root",
    "label":  "Pester",
    "children":  [

                 ]
}

[2021-05-03 21:01:42.028] [DEBUG] child process exited with code 0
[2021-05-03 21:01:42.031] [ERROR] Unable to parse JSON data from Pester test discovery script. Contents: {
    "type":  "suite",
    "id":  "root",
    "label":  "Pester",
    "children":  [

                 ]
}

[2021-05-03 21:01:46.327] [INFO] Running Pester tests ["root"]

I see the Import-Module Pester errors so I gran a Get-Command *pester* and I see two different 'Invoke-Pester' commands with different versions but when I run Get-Command -Module Pester I am only seeing version 5.1.1.

image

image

I'm not sure how this happened but this might be part of the problem.

TylerLeonhardt commented 3 years ago

It's possible... in the log is spits out the full script that it's running:

$Path = @(
    'c:\SWDev\PowerShell\MerlinNet\Tests\Invoke-MNetSqlQuery.Tests.ps1'
)

$VerbosePreference = 'Ignore'
$WarningPreference = 'Ignore'
$DebugPreference = 'Ignore'
Import-Module Pester -MinimumVersion 5.0.0 -ErrorAction Stop
function Discover-Test
{
    [CmdletBinding()]
    param(
        [Parameter(Mandatory)]
        [String[]] $Path,
        [String[]] $ExcludePath
    )
    & (Get-Module Pester) { 
        param (
            $Path, 
            $ExcludePath,
            $SessionState)

        Reset-TestSuiteState
        # to avoid Describe thinking that we run in interactive mode
        $invokedViaInvokePester = $true
        $files = Find-File -Path $Path -ExcludePath $ExcludePath -Extension $PesterPreference.Run.TestExtension.Value
        $containers = foreach ($f in $files) {
            <# HACK: We check to see if there is a single Describe block in the file so that we don't accidentally execute code that shouldn't need to be executed. #>
            if (!(Select-String -Path $f -SimpleMatch 'Describe')) {
                continue
            }
            New-BlockContainerObject -File (Get-Item $f)
        }
        Find-Test -BlockContainer $containers -SessionState $SessionState } -Path $Path -ExcludePath $ExcludePath -SessionState $PSCmdlet.SessionState
}

function New-SuiteObject ($Block) { 
    [PSCustomObject]@{
        type = 'suite'
        id = $Block.ScriptBlock.File + ';' + $Block.StartLine
        file = $Block.ScriptBlock.File
        line = $Block.StartLine - 1
        label = $Block.Name
        children = [Collections.Generic.List[Object]]@()
    }
}

function New-TestObject ($Test) { 
    [PSCustomObject]@{
        type = 'test'
        id = $Test.ScriptBlock.File + ';' + $Test.StartLine
        file = $Test.ScriptBlock.File
        line = $Test.StartLine - 1
        label = $Test.Name
    }
}

function fold ($children, $Block) {
    foreach ($b in $Block.Blocks) { 
        $o = (New-SuiteObject $b)
        $children.Add($o)
        fold $o.children $b
    }

    $hashset = [System.Collections.Generic.HashSet[string]]::new()
    foreach ($t in $Block.Tests) {
        $key = "$($t.ExpandedPath):$($t.StartLine)"
        if ($hashset.Contains($key)) {
            continue
        }
        $children.Add((New-TestObject $t))
        $hashset.Add($key) | Out-Null
    }
    $hashset.Clear() | Out-Null
}

$found = Discover-Test -Path $Path

# whole suite
$suite = [PSCustomObject]@{
    Blocks = [Collections.Generic.List[Object]] $found
    Tests = [Collections.Generic.List[Object]]@()
}

$testSuiteInfo = [PSCustomObject]@{
    type = 'suite'
    id = 'root'
    label = 'Pester'
    children = [Collections.Generic.List[Object]]@()
}

foreach ($file in $found) {
    $fileSuite = [PSCustomObject]@{
        type = 'suite'
        id = $file.BlockContainer.Item.FullName
        file = $file.BlockContainer.Item.FullName
        label = $file.BlockContainer.Item.Name
        children = [Collections.Generic.List[Object]]@()
    }
    $testSuiteInfo.children.Add($fileSuite)
    fold $fileSuite.children $file
}

$testSuiteInfo | ConvertTo-Json -Depth 100

Can you try to take that one line at a time in a fresh PowerShell window and tell me if anything looks odd?

obsidian33 commented 3 years ago

Nothing looked odd to me. No errors or exceptions. This is the json that was generated:

{
    "type": "suite",
    "id": "root",
    "label": "Pester",
    "children": [{
            "type": "suite",
            "id": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1",
            "file": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1",
            "label": "Invoke-MNetSqlQuery.Tests.ps1",
            "children": [{
                    "type": "suite",
                    "id": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1;5",
                    "file": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1",
                    "line": 4,
                    "label": "Invoke-MNetSqlQuery",
                    "children": [{
                            "type": "suite",
                            "id": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1;6",
                            "file": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1",
                            "line": 5,
                            "label": "When not connected to network",
                            "children": [{
                                    "type": "test",
                                    "id": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1;7",
                                    "file": "C:\\SWDev\\PowerShell\\MerlinNet\\Tests\\Invoke-MNetSqlQuery.Tests.ps1",
                                    "line": 6,
                                    "label": "Should throw an exception"
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ]
}

The organization of the children do seem a bit odd to me though since there is only one describe block that contains the 3 context.

BeforeAll {
    . $PSScriptRoot\_InitializeTests.ps1
}

Describe "Invoke-MNetSqlQuery" {
    Context "When not connected to network" -Tag "Network" {
        It "Should throw an exception" {
            # ...
        }
    }

    Context "When an invalid SQL query is supplied" {
        It "Should throw an exception" {
            # ...
        }
    }

    Context "When query returns zero results" {
        It "Should return an empty DataRow object" {
            # ....
        }

    }
}
obsidian33 commented 3 years ago

I was switching between PowerShell Core and Windows PowerShell and as I was doing so this error popped up in the 'Pester Explorer Log'.

[2021-05-07 06:11:49.476] [ERROR] stderr: Reset-TestSuiteState: 

[2021-05-07 06:11:49.477] [ERROR] stderr: Line |
  25 |          Reset-TestSuiteState
     |          ~~~~~~~~~~~~~~~~~~~~
     | The term 'Reset-TestSuiteState' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

I took the liberty of re-running the script output in the 'Pester Explorer Log' and I'm seeing the command not found exception for both Windows PowerShell and PowerShell Core now.

Reset-TestSuiteState : The term 'Reset-TestSuiteState' is not recognized as the name of a cmdlet, function, script 
file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is 
correct and try again.
At C:\Scripts\debug-vscode-adapter.ps1:24 char:9
+         Reset-TestSuiteState
+         ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Reset-TestSuiteState:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException