Winforms VB application with Single Instance setting crashes for multiple users on terminal server

[Olina-Zhang]

.NET version

.Net 9.0 SDK: 9.0.100-preview.4.24217.27

Did it work in .NET Framework?

Yes

Did it work in any of the earlier releases of .NET Core or .NET 5+?

No, repro in .Net 8

Issue description

Creating a Winforms .Net 9 or 8 VB application with setting "single instance application" to true, put it on a terminal server, more users with permission to work on this server using RDP. The first user starts the program just fine, the second user start the same app and it crashes on startup.

Here is the log when it is reproduced:

Application: VBSingleInstance_NET9.exe
CoreCLR Version: 9.0.24.21505
.NET Version: 9.0.0-preview.4.24215.5
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.VisualBasic.ApplicationServices.CantStartSingleInstanceException: This single-instance application could not connect to the original instance.
   at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(String[] commandLine)
   at VBSingleInstance_NET9.My.MyApplication.Main(String[] Args) in :line 83

Steps to reproduce

1. Create a Winforms VB .NET application
2. In Solution Explorer window, show all files, find Application.Designer.vb file, set IsSingleInstance to true
3. Build this application
4. Put this applicaiton to a prepared server machine with multiple users access
5. First user to log in this terminal server machine by RDP, launch this application
6. Second user to log in this terminal server machine by RDP, launch this application too

MoreInfo: this is a customer feedback: https://developercommunity.visualstudio.com/t/NET-8-single-instance-winforms-crashes-/10634133

[elachlan]

Duplicate of #3715

[elachlan]

@Olina-Zhang the exception is probably caused by the fact that the named pipe uses PipeOptions.CurrentUserOnly. So it can't connect to other users named pipes. Which is what we want. My suggested fix in #11258 should mitigate this issue.

[merriemcgaw]

@KlausLoeffelmann would you be able to look at the changes? @elachlan, thank you so much for the PR!

We'll want to make sure that security (@Shyam-Gupta and @GrabYourPitchforks ) are also happy the change.