microsoft / go-winio

Win32 IO-related utilities for Go
MIT License
950 stars 182 forks source link

Unable to create named pipe on certain WS2012, Win 10 Pro machines #280

Closed rcarman-r7 closed 1 year ago

rcarman-r7 commented 1 year ago

In a lab setting of multiple Windows machines, several machines were unable to create a named pipe server:

This issue was solved by modifying the ObjectAttributes.Attributes flag being passed to NtCreateNamedPipeFile. This change did not impact previously working machines so it appears safe for general use.

Background: Inspecting the output from the system call, ntCreateNamedPipeFile() returned the following: image Retval: 3221225530 Status: -1073741766 There was no named pipe present in the PowerShell output: Get-ChildItem -Path "\\.\pipe\"

To rule out machine specific issues, an alternate named pipe server was built from the Named Pipe C++ example from Microsoft: https://learn.microsoft.com/en-us/windows/win32/ipc/multithreaded-pipe-server Notably, this example uses CreateNamedPipeW versus NtCreateNamedPipeFile. The named pipe was successfully created with retval=0, and the pipe was observed in the PowerShell output: Get-ChildItem -Path "\\.\pipe\"

Now, there were 2 executables with different retvals from the same machine. Knowing that CreateNamedPipeW eventually calls NtCreateNamedPipeFile, WinDBG was used to trace the call. The outcome of the investigation was that CreateNamedPipeW was passing a 0x40 in ObjectAttributes.Attributes flag, whereas go-winio was passing 0x0. The issue was resolved by setting ObjectAttributes.Attributes to 0x40, corresponding to OBJ_CASE_INSENSITIVE.

helsaawy commented 1 year ago

overall this looks good, DCO task happy you will have to sign your commits: git rebase HEAD~7 --signoff may work, or you squash all the commits and sign that