Whitespace Between Parameters Is Trimming Required Quotes

sheldonhull commented 4 years ago

✅ Make sure you are able to repro it on the latest released version
✅ Perform a quick search for existing issues to check if this bug has already been reported.
- Might be related to #1540

This was one of the more simple examples I could find. I found this issue popping up in random files and finally realized it was a formatter issue.

Steps to reproduce

Describe "DescribeName" {
    Context "ContextName" {
        It "ItName" {
            Assertion
        }
        It "output: attributes     matches  $($outputs.attributes)" {
            $results.attributes | Should -Be $outputs.attributes
        }
    }
}

Expected behavior

Describe "DescribeName" {
    Context "ContextName" {
        It "ItName" {
            Assertion
        }
        It "output: attributes     matches  $($outputs.attributes)" {
            $results.attributes | Should -Be $outputs.attributes
        }
    }
}

Actual behavior

Describe "DescribeName" {
    Context "ContextName" {
        It "ItName" {
            Assertion
        }
        It "output: attributes     matches  $($outputs.attributes){ <----- it removed valid quote, resulting in breaking syntax
            $results.attributes | Should -Be $outputs.attributes
        }
    }
}

Environment data


> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.0.3
PSEdition                      Core
GitCommitId                    7.0.3
OS                             Darwin 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.19.0
1.18.3
1.19.1

rjmholt commented 4 years ago

@sheldonhull what's the output of Get-Module PSScriptAnalyzer here? You have 1.19.1 installed, but is that the one that's loaded and running?

sheldonhull commented 4 years ago

@rjmholt the PowerShell Editor Services extension diagnostic level logs pointed this out:

[Info  - 2:09:31 PM] Microsoft.PowerShell.EditorServices.Services.Analysis.PssaCmdletAnalysisEngine: PSScriptAnalyzer successfully imported:
    Version: 1.19.1
    Exported Cmdlets:
    Get-ScriptAnalyzerRule
    Invoke-Formatter
    Invoke-ScriptAnalyzer

get-installedmodule psscriptanalyzer |FL provided this

Name                       : PSScriptAnalyzer
Version                    : 1.19.0
Type                       : Module
Author                     : Microsoft Corporation
CompanyName                : PowerShellTeam JamesTruher-MSFT
Copyright                  : (c) Microsoft Corporation 2016. All rights reserved.
PublishedDate              : 5/4/2020 5:48:56 PM
InstalledDate              : 6/23/2020 6:59:51 PM

I think it has something to do with the variable int he string for "it". For instance, if you change the value to:

It "output attributes matches outputs.attributes"

no issue will occur. As soon as you add the $outputs it begins to fail. Does this provide more useful context?

rjmholt commented 4 years ago

@sheldonhull how about Get-Module PSScriptAnalyzer? Get-InstalledModule isn't going to help here.

Is the formatting issue you're seeing something occurring when formatting through the extension or with Invoke-Formatter? Are you able to reproduce with just Invoke-Formatter in the console?

sheldonhull commented 4 years ago

I was using the Format Document command in VSCode. I just ran manual import and it reported:

VERBOSE: Loading module from path '/Users/sheldonhull/.local/share/powershell/Modules/PSScriptAnalyzer/1.19.0/PSScriptAnalyzer.psd1'.

When running Invoke-Formatter I was able to narrow down the option that seems to be causing this for me.

Reproduce via Run Selection

```powershell Import-Module PSScriptAnalyzer -RequiredVersion 1.19.1 Write-Host "PSScriptAnalyzer Version: $((Get-Module PSScriptAnalyzer).Version.ToString())" $script = @' Describe "DescribeName" { Context "ContextName" { It "ItName" { Assertion } It "output: attributes matches $($outputs.attributes)" { $results.attributes | Should -Be $outputs.attributes } } } '@ $Rules = @{ IncludeRules = @( 'PSPlaceOpenBrace', 'PSPlaceCloseBrace', 'PSUseConsistentWhitespace', 'PSUseConsistentIndentation', 'PSAlignAssignmentStatement', 'PSUseCorrectCasing' ) Rules = @{ PSPlaceOpenBrace = @{ Enable = $true OnSameLine = $true NewLineAfter = $true IgnoreOneLineBlock = $true } PSPlaceCloseBrace = @{ Enable = $true NewLineAfter = $true IgnoreOneLineBlock = $true NoEmptyLineBefore = $false } PSUseConsistentIndentation = @{ Enable = $true Kind = 'space' PipelineIndentation = 'IncreaseIndentationForFirstPipeline' IndentationSize = 4 } PSUseConsistentWhitespace = @{ Enable = $true CheckInnerBrace = $true CheckOpenBrace = $true CheckOpenParen = $true CheckOperator = $true CheckPipe = $true CheckPipeForRedundantWhitespace = $true CheckSeparator = $true CheckParameter = $false } PSAlignAssignmentStatement = @{ Enable = $true CheckHashtable = $true } PSUseCorrectCasing = @{ Enable = $true } } } Invoke-Formatter -ScriptDefinition $script -Settings $Rules ```

rjmholt commented 4 years ago

Reproduce via Run Selection

I'm assuming the output of this reproduces your issue. Well done on working out the settings btw — we really need a simple way to convert those from VSCode to PSSA form.

If you can try in a fresh console with Import-Module PSScriptAnalyzer -RequiredVersion 1.19.1 and then your repro, just let me know if that also reproduces the issue.

sheldonhull commented 4 years ago

I just installed that version, and then ran the statement with CheckParameter = $true and no issue now. Did you figure it out?

I just searched the repo and found the commit :-) Thanks for the kind words. Let me know if you need anything else. 🥂

rjmholt commented 4 years ago

I just searched the repo and found the commit :-) Thanks for the kind words.

Ah ok, so does that mean it's fixed in 1.19.1? If so that means you're actually experiencing https://github.com/PowerShell/vscode-powershell/issues/2697

sheldonhull commented 4 years ago

I'll have to revisit again tomorrow. I tried manually as we talked about it and it didn't seem to have an issue, but yet the editorservices issue doesn't explain that as it showed it did load 1.19.1 in the comments above. Am I missing something else?

rjmholt commented 4 years ago

editorservices issue doesn't explain that as it showed it did load 1.19.1 in the comments above. Am I missing something else?

Hmmm, yeah that's true. Two possibilities are:

PSES configurations aren't hooked up correctly (but then, there's no "please omit random quotes from my script" configuration)
The initial runspace loaded in PSES for PSSA has the correct PSSA version, but a later one in the runspace pool loads the wrong version

In any case, this seems to not occur with just PSSA, so I think https://github.com/PowerShell/vscode-powershell/issues/2697 is the best place for us to continue investigating

ninmonkey commented 4 years ago

@rjmholt this seems to not occur with just PSSA

If Invoke-ScriptAnalyzer is returning multiple ParserErrors, shouldn't Invoke-ScriptFormatter exit without attempting to format? That combined with certain flags seems to trigger the code deletion bugs.

When I run Invoke-ScriptAnalyzer it gives parse errors. If PSUseConsistentWhitespace.CheckParameter = True the formatter breaks on this test case.

RuleName                          Severity Message
--------                          -------- -------
MissingArgument                 ParseError Missing argument in parameter list.
TerminatorExpectedAtEndOfString ParseError The string is missing the terminator: ".
MissingEndCurlyBrace            ParseError Missing closing '}' in statement block or type definition.

I ran pwsh outside of vs code then imported the module.

Test case

$Original = @'
Function BadCode {
    "stuff" | Measure-Object" 'ps1'

    $InputList | ForEach-Object {
    } | Select-Object -First 2
    | Join-String -sep ", " -OutputPrefix 'Results: '
}
'@

$settings = @{
    IncludeRules = @(
        "PSUseConsistentWhitespace"
    )
    Rules        = @{
        PSUseConsistentWhitespace = @{
            Enable         = $True
            CheckParameter = $false
        }
    }
}

$out1 = Invoke-Formatter -ScriptDefinition $Original -Settings $settings

$settings.rules.PSUseConsistentWhitespace.CheckParameter = $True
$out2 = Invoke-Formatter -ScriptDefinition $Original -Settings $settings

$out1
'+' * 30
$out2

if ($out1 -ne $out2) {
    Write-Error 'formatting does not match'
}

Expected

Function BadCode {
    "stuff" | Measure-Object" 'ps1'

    $InputList | ForEach-Object {
    } | Select-Object -First 2
    | Join-String -sep ", " -OutputPrefix 'Results: '
}

Actual

Function BadCode {
    "stuff" | Measure-Object" -OutputPrefix 'Results: '
}

If you change the line from Object" 'ps1' to Object " 'ps1' it's still invalid code, but the formatter no longer mutates it.

"stuff" | Measure-Object " 'ps1'

system

> $psEditor | select EditorServicesVersion

> $PSVersionTable | ft

Name                           Value
----                           -----
PSVersion                      7.0.3
PSEdition                      Core
GitCommitId                    7.0.3
OS                             Microsoft Windows 10.0.19041
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> (Get-Module -ListAvailable PSScriptAnalyzer).Version | ForEach-Object { $_.ToString() }
1.19.1

Velocet commented 4 years ago

I got the same problem with the following line (and many other .. this is just an example) in script not written by me: Write-Host ($Error[0]).Exception this gets formatted to this if "Whitespace Between Parameters" is switched on: Write-Host ($Error[0]Exception

Visual Studio Code v1.48.2 64-bit PowerShell Extension v2020.6.0 PowerShell runtime version: 5.1.18362.752, edition: Desktop

Also: the extension doesn't care that i don't want to load my profile scripts .. i really dunno what happened to the PowerShell extension in the last few months but i gets worse with every release. I just no possible to work with PwSh and VSCode anymore. There so many bugs that i think i have to abandon it. I was using VSCode as my main PwSh development tool since the very beginning of VSCode but on my private machine i get constant errors and on my work machine i have so many bugs that i stopped counting them. Thanks for all your work and everything and it's great that all of this is for free .. but all those bugs frustrates me so much that i disabled the PwSh extension (heck.. just look at the new -parallel foreach option which consumes a ton of RAM cause the GC seems not able to catch it .. how is it possible to not catch such a bug or at least make a note about it if it's a 'feature'?).

bergmeister commented 4 years ago

This bug was fixed in PSSA 1.19.1 but the stable PowerShell extension is not shipping that yet (not sure why, will chase that). If you use the preview extension and restart vs-code then this should not happen any more

jansohn commented 3 years ago

This bug was fixed in PSSA 1.19.1 but the stable PowerShell extension is not shipping that yet (not sure why, will chase that). If you use the preview extension and restart vs-code then this should not happen any more

The Powershell Extension still seems to ship with an old version as I also experience bugs when turning that parameter on. How can I check which version is bundled with the Powershell extension?

PowerShell / PSScriptAnalyzer