Null reference exceptions when trying to complete variable names

[ghost]

Prerequisites

• [X] Write a descriptive title.
• [X] Make sure you are able to repro it on the latest version
• [X] Search the existing issues.

Steps to reproduce

I have PowerShell Editor Services configured in KDE Kate using a script that does essentially:

& 'C:/Tools/PowerShellEditorServices/PowerShellEditorServices/Start-EditorServices.ps1' -BundledModulesPath C:/Tools/PowerShellEditorServices -LogPath $logPath -SessionDetailsPath $sessionPath -FeatureFlags @() -AdditionalModules @() -Stdio -LogLevel Normal -HostName $host -HostProfileId $profile -HostVersion $hostVersion

Sometimes, when I am typing a variable reference, PowerShell Editor Services throws a null reference exception. When it happens, it happens repeatedly, but I don't yet know the exact conditions. I will try to find specific trigger conditions tomorrow.

Expected behavior

Either a completion is offered successfully or fails silently.

Actual behavior

An exception is thrown, which is displayed in the editor window distractingly. (In Kate this is especially bad because the Output window pops up, grabbing keyboard focus.)

Error details

OmniSharp.Extensions.JsonRpc.DefaultRequestInvoker: Failed to handle request textDocument/completion 1727 - System.NullReferenceException: Object reference not set to an instance of an object.
      at Microsoft.PowerShell.EditorServices.Services.Symbols.AstOperations.GetCompletionsAsync(Ast scriptAst, Token[] currentTokens, Int32 fileOffset, IInternalPowerShellExecutionService executionService, ILogger logger, CancellationToken cancellationToken) in D:\a\_work\1\s\src\PowerShellEditorServices\Services\Symbols\Vistors\AstOperations.cs:line 141
      at Microsoft.PowerShell.EditorServices.Handlers.PsesCompletionHandler.GetCompletionsInFileAsync(ScriptFile scriptFile, Int32 lineNumber, Int32 columnNumber, CancellationToken cancellationToken) in D:\a\_work\1\s\src\PowerShellEditorServices\Services\TextDocument\Handlers\CompletionHandler.cs:line 248
      at Microsoft.PowerShell.EditorServices.Handlers.PsesCompletionHandler.Handle(CompletionParams request, CancellationToken cancellationToken) in D:\a\_work\1\s\src\PowerShellEditorServices\Services\TextDocument\Handlers\CompletionHandler.cs:line 82
      at OmniSharp.Extensions.LanguageServer.Server.Pipelines.SemanticTokensDeltaPipeline`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at OmniSharp.Extensions.LanguageServer.Server.Pipelines.ResolveCommandPipeline`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestPreProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestPostProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestExceptionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestExceptionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestExceptionActionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at MediatR.Pipeline.RequestExceptionActionProcessorBehavior`2.Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate`1 next)
      at OmniSharp.Extensions.JsonRpc.RequestRouterBase`1.<RouteRequest>g__InnerRoute|7_0(IServiceScopeFactory serviceScopeFactory, Request request, TDescriptor descriptor, Object params, CancellationToken token, ILogger logger)
      at OmniSharp.Extensions.JsonRpc.RequestRouterBase`1.RouteRequest(IRequestDescriptor`1 descriptors, Request request, CancellationToken token)
      at OmniSharp.Extensions.JsonRpc.DefaultRequestInvoker.<>c__DisplayClass10_0.<<RouteRequest>b__5>d.MoveNext() | Method='textDocument/completion' RequestId='1727'

Environment data

PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.2.6
PSEdition                      Core
GitCommitId                    7.2.6
OS                             Microsoft Windows 10.0.19043
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Version

3.5.1

Visuals

No response

[ghost]

I'm sorry, after a few hours I'm giving up on producing a useful example case. It seems like this only happens in large and complex scripts with lots of defined variables, which if true means a minimal example is literally impossible. It definitely happens on a textDocument/completion call but we already knew that.

After some wrangling I got tee capturing of both input and output set up (this is surprisingly difficult to do on Windows) in the hopes that a trimmed-down canned input could be used to trigger the problem, but alas, feeding canned input to PSES appears to be a non-trivial job that requires specific timing.

[ghost]

I un-gave up and took a different approach and I now seem to have a reasonably-sized script that triggers it, and a decent log of the input from Kate for the entire session. Attached are (privacy sanitized) stdin and stdout from PSES, plus the script file (renamed with an additional .txt suffix to bypass GitHub restrictions).

[ghost]

The error triggers upon typing the $ for the value of key, so the easiest way to see it if you can in fact reproduce is to delete it, then save the file and type it again.

[andyleejordan]

Hey, I was able to repro this! Interestingly the VS Code client doesn't crash after that fails, but the null reference exception is certainly happening, so we can hopefully get this fixed ASAP. Thank you so much for the detailed repro info!!! 🥳🥳🥳

[ghost]

@andschwa Ah, excellent to know it reproduces on VS Code! I guess the VS Code extension just doesn't complain about exceptions as loudly as Kate's LSP client.

[andyleejordan]

Fixed with a regression test!

[MartinGC94]

The completion code shouldn't throw errors so it's probably worth creating an issue in https://github.com/PowerShell/PowerShell to fix the underlying issue.

[andyleejordan]

Thanks so much @MartinGC94!

[andyleejordan]

Hi @alexbuzzbee! We just got this fixed in v3.6.0, can you verify for us?

[ghost]

Well, I don't seem to be able to reproduce it anymore and it looks like the underlying issue has a fix committed too. Seems good to me. Thanks!