Start-EditorServices.ps1 terminates on first request when started from Emacs (eglot)

macmacy commented 4 months ago

Prerequisites

[X] I have written a descriptive issue title.
[X] I have searched all open and closed issues to ensure it has not already been reported.
[X] I have read the troubleshooting guide.
[X] I am sure this issue is with PowerShell Editor Services itself and does not reproduce in a standalone PowerShell instance, and is not an issue with my editor.
[X] I have verified that I am using the latest version of PowerShell Editor Services.
[X] If this is a security issue, I have read the security issue reporting guidance.

Summary

I use Emacs+eglot with other language servers: OmniSharp, pyright, clangd, but cannot make it work with PowerShellEditorServices. The services command starts but exists after a while. Last message in log is Could not write to output handler, perhaps serialization failed?.

PowerShell Version

PS > $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Editor Version

GNU Emacs 29.1 (build 2, x86_64-w64-mingw32) of 2023-08-02

PowerShell Editor Services Version

3.17.0 (latest)

Steps to Reproduce

I have PSES configured in .emacs like that:

(add-to-list
 'eglot-server-programs
 `(powershell-mode
       . ("pwsh" "-NoLogo" "-NoProfile" "-Command" , "c:/bin/PowerShellEditorServices/PowerShellEditorServices/Start-EditorServices.ps1"
          "-HostName" "Emacs" "-HostProfileId" "Emacs" "-HostVersion" "1.0.0"
          "-BundledModulesPath", "c:/bin/PowerShellEditorServices"
          "-LogPath" ,"c:/users/mmaczynski/AppData/Roaming/.pses-session/logs.log" "-LogLevel" "Diagnostic"
          "-SessionDetailsPath" ,"c:/users/mmaczynski/AppData/Roaming/.pses-session/session.json"
          "-Stdio")))

(add-hook 'powershell-mode 'eglot-ensure)

When I load .ps1 file, or execute M-x eglot when in powershell-mode, the Start-EditorServices is started but exists after a while, logging the following in session log file:

2024-02-28 14:18:59.973 +01:00 [VRB] Could not write to output handler, perhaps serialization failed?
System.Threading.Channels.ChannelClosedException: The channel has been closed.
   at System.Threading.Channels.AsyncOperation`1.GetResult(Int16 token)
   at OmniSharp.Extensions.JsonRpc.OutputHandler.ProcessOutputStream(CancellationToken cancellationToken)

I guess the server process exits on my first attempt to edit a file e.g. starting to type "Get-" or executing M-x eglot-rename

Visuals

No response

Logs

log files from PSES session directory: logs.log StartEditorServices.log

The content of Emacs eglot events buffer: eglotlog.txt

andyleejordan commented 4 months ago

Hi there, sorry for the complicated setup, but what I believe is the problem is your argument to -BundledModulesPath, you're giving it the path to the root of the project "c:/bin/PowerShellEditorServices" but it needs to be the path to the bundled modules inside the project (the folder with PSReadLine, PSScriptAnalyzer, etc.) at "c:/bin/PowerShellEditorServices/module". Here's the definition for the end-to-end test using Eglot: https://github.com/PowerShell/PowerShellEditorServices/blob/main/test/emacs-test.el

andyleejordan commented 4 months ago

FWIW I'm trying to simplify this setup for you: https://github.com/PowerShell/PowerShellEditorServices/pull/2129 but I need to get back to this PR.

macmacy commented 4 months ago

Thank you for your fast response. Your suggestion was correct: after changing -BundledModulesPath it started to work. Awesome! Thank you very much. However, the problem is probably not only my misconfiguration. Initially I used PSES zipped release binaries, not repository. The directory structure was like that (release ZIP unpacked):

C:\BIN\POWERSHELLEDITORSERVICES
├───docs
├───Plaster
├───PowerShellEditorServices
├───PSReadLine
├───PSScriptAnalyzer
└───_manifest

This is why I set up bundled modules path to c:/bin/powershelleditorservices - this is where modules seems to be, and there is no modules subdir here. And this does not work. However, when I built from sources, my directory structure is like that:

C:\BIN\POWERSHELLEDITORSERVICES
├───.github
├───.vsts-ci
├───docs
├───module
│   ├───docs
│   ├───Plaster
│   ├───PowerShellEditorServices
│   ├───PSReadLine
│   └───PSScriptAnalyzer
├───src 
└───tools

With such structure and setting -BundledModulesPath to "c:/bin/PowerShellEditorServices/module" - it works. Which seems a bit strange to me, as the structure under C:\BIN\POWERSHELLEDITORSERVICES in first case looks same as under C:\BIN\POWERSHELLEDITORSERVICES\module in second case. However - something makes difference here. Anyway - with project built from sources and parameters set up as in emacs-test.el - it works fine. Thanks once again!

andyleejordan commented 4 months ago

Well...that's strange. I'll take a look.

andyleejordan commented 4 months ago

Well this encouraged me to get back to https://github.com/PowerShell/PowerShellEditorServices/pull/2129; when we get this released would you test it out for us?

macmacy commented 4 months ago

Absolutely - just drop me a note here.

andyleejordan commented 4 months ago

@macmacy it's out if you want to give it a try! https://github.com/PowerShell/PowerShellEditorServices/releases/tag/v3.18.0

github-actions[bot] commented 4 months ago

This issue has been labeled as resolved, please verify the provided fix (or other reason).

macmacy commented 3 months ago

I was trying to test it with v3.18.0, but it looks like PS files were signed in different way and current certificate is not accepted on my system. It was not the issue with previous release. This is how it looks like now:

PS C:\bin\_PowerShellEditorServices> Get-AuthenticodeSignature  .\PowerShellEditorServices\Start-EditorServices.ps1|fl

SignerCertificate      : [Subject]
                           CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

                         [Issuer]
                           CN=Microsoft Code Signing PCA 2011, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

                         [Serial Number]
                           33000003AE2D3551C8538F551D0000000003AE

                         [Not Before]
                           16.11.2023 20:08:59

                         [Not After]
                           14.11.2024 20:08:59

                         [Thumbprint]
                           F9A7CF9FBE13BAC767F4781061332DA6E8B4E0EE

TimeStamperCertificate : [Subject]
                           CN=Microsoft Time-Stamp Service, OU=nShield TSS ESN:9600-05E0-D947, OU=Microsoft America Operations, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

                         [Issuer]
                           CN=Microsoft Time-Stamp PCA 2010, O=Microsoft Corporation, L=Redmond, S=Washington, C=US

                         [Serial Number]
                           33000001EF893F56A158CCA8DA0001000001EF

                         [Not Before]
                           06.12.2023 19:45:48

                         [Not After]
                           05.03.2025 19:45:48

                         [Thumbprint]
                           4B708F358316DE6B5131DA615B5F1EDC93ED20FF

Status                 : NotTrusted
StatusMessage          : File C:\bin\_PowerShellEditorServices\PowerShellEditorServices\Start-EditorServices.ps1 is signed but the signer is not trusted on this system.
Path                   : C:\bin\_PowerShellEditorServices\PowerShellEditorServices\Start-EditorServices.ps1
SignatureType          : Authenticode
IsOSBinary             : False

At explorer, it looks like that: bad-cert

seems that there is something wrong with this certificate indeed.

github-actions[bot] commented 3 months ago

This issue has been labeled as resolved, please verify the provided fix (or other reason).

andyleejordan commented 3 months ago

Hm, I'm going to suggest redownloading to ensure the integrity of the bits, and check your system time? Did you download this exact asset https://github.com/PowerShell/PowerShellEditorServices/releases/download/v3.18.0/PowerShellEditorServices.zip? I just downloaded the release form GitHub and inspected the certificate, it looks correct and not revoked:

andyleejordan commented 3 months ago

Here's the SHA256 sum:

> certutil -hashfile "~\Downloads\PowerShellEditorServices.zip" sha256
SHA256 hash of ~\Downloads\PowerShellEditorServices.zip:
b5624eeae84e4a23e1ef2b9516d29a9e0df02640470987a9f95757c05faee926
CertUtil: -hashfile command completed successfully.

macmacy commented 3 months ago

In short words: I can confirm it works now. With the following setup in .emacs:

(add-to-list
 'eglot-server-programs
 `(powershell-mode
       . ("pwsh" "-NoLogo" "-NoProfile" "-Command" , 
          "c:/bin/PowerShellEditorServices/PowerShellEditorServices/Start-EditorServices.ps1"
          "-HostName" "Emacs" "-HostProfileId" "Emacs" "-HostVersion" "1.0.0"
          "-BundledModulesPath", "c:/bin/PowerShellEditorServices/PowerShellEditorServices"
          "-LogPath" ,"c:/users/mysername/AppData/Roaming/.pses-session/logs.log" "-LogLevel" "Diagnostic"
          "-SessionDetailsPath" ,"c:/users/myusername/AppData/Roaming/.pses-session/session.json"
          "-Stdio")))

I'm sorry for a headache with certificate.. This was my local problem. As for excuse, let me explain (partially) what was happening. I was using exactly same zip as you've mentioned in the comment above, SHA256 matches. While Get-AuthenticodeSignature and Windows explorer keep claiming problem, checking extracted cert with certutil shows it is valid and not revoked.

I tried it on another system: no problems with signing certificate (yes, I know this is the first thing I should do...) Finally, I found this certificate in my Windows personal certificate store under "Untrusted Certificates" store! Being certified by Microsoft Root Certification Authority. It was one and only certificate there. I have no faintest idea HOW it got there... I did not put it there in any conscious way. What I googled out, is that this is Windows "blacklist" for certs and that it is Windows updates who writes this store. I also found some ancient article from 2012 about "Microsoft certification authority signing certificates added to the Untrusted Certificate Store": https://msrc.microsoft.com/blog/2012/06/microsoft-certification-authority-signing-certificates-added-to-the-untrusted-certificate-store/. So - such things happened. But nothing up to date.

Anyway, after removing it from there - I got it working without any more complaints. Thanks once again for your support!

andyleejordan commented 3 months ago

Awesome! Just FYI with the simplification, I think you can just run this code now to setup (note the removal of most of the silly unnecessary arguments:

(add-to-list
 'eglot-server-programs
 `(powershell-mode
       . ("pwsh" "-NoLogo" "-NoProfile" "-Command" , 
          "c:/bin/PowerShellEditorServices/PowerShellEditorServices/Start-EditorServices.ps1"
          "-LogPath" ,"c:/users/mysername/AppData/Roaming/.pses-session/logs.log" "-LogLevel" "Diagnostic"
          "-SessionDetailsPath" ,"c:/users/myusername/AppData/Roaming/.pses-session/session.json"
          "-Stdio")))

Other than I'm unsure about which commas belong where...

github-actions[bot] commented 3 months ago

This issue has been labeled as resolved, please verify the provided fix (or other reason).

github-actions[bot] commented 3 months ago

This issue has been labeled as resolved, please verify the provided fix (or other reason).

github-actions[bot] commented 3 months ago

This issue has been labeled as needing fix verification and has not had any activity a week. It has been closed for housekeeping purposes.

PowerShell / PowerShellEditorServices