PSExec works, but PAExec returns Error code 5 "Access Denied" when target is on different subnet.

[ghost]

I have been troubleshooting connection issues with an Ansible module that uses pypsexec (a python implementation of PAExec code), but have been getting Access Denied on most hosts but not all. From what I can tell, as long as the remote host is on the same subnet as the localhost, there is no issue. If I try connecting to another subnet, even if it's physically connected to the same network switch, the connection tends to fail when starting the PAExec service. At first I thought I missed something in the prerequisite security settings, but then everything works just fine from PSExec. I have tried moving a working instance to another subnet, and it continued to work with PAExec, so I don't know if having prior success leads to continued success. Will continue to troubleshoot and see if I can narrow this down a little more, any tips or ways to enable additional verbose/debug logging would be appreciated.

c:\Temp>psexec.exe /AcceptEula \\foo.contoso.com cmd.exe

PsExec v2.33 - Execute processes remotely
Copyright (C) 2001-2021 Mark Russinovich
Sysinternals - www.sysinternals.com

Microsoft Windows [Version 10.0.19041.804]
(c) 2020 Microsoft Corporation. All rights reserved.

C:\Windows\system32>hostname
foo

C:\Windows\system32>exit
cmd.exe exited on foo.contoso.com with error code 0.

c:\Temp>paexec.exe /AcceptEula \\foo.contoso.com cmd.exe

PAExec v1.28 - Execute Programs Remotely
Copyright (c) 2012-2019 Power Admin LLC
www.poweradmin.com/PAExec

Connecting to foo.contoso.com...
Starting PAExec service on foo.contoso.com...
Failed to start service on foo.contoso.com Access is denied. [Err=0x5, 5]
Failed to stop PAExec service. The service has not been started. [Err=0x426, 1062]

PAExec returning exit code -6

c:\Temp>paexec.exe /AcceptEula \\bar.contoso.com cmd.exe

PAExec v1.28 - Execute Programs Remotely
Copyright (c) 2012-2019 Power Admin LLC
www.poweradmin.com/PAExec

Connecting to bar.contoso.com...
Starting PAExec service on bar.contoso.com...

Microsoft Windows [Version 10.0.19042.870]
(c) 2020 Microsoft Corporation. All rights reserved.

C:\Windows\system32>hostname
bar

C:\WINDOWS\system32>exit
cmd.exe returned 0

PAExec returning exit code 0

Confirmed that PAExec works from subnet A to A with 100% of available computers to test. PAExec has failed at least 90% of the time with computers I've tested from subnet A to B with Access Denied, but the same tests succeed 100% of the time from PSExec. Sample size has been a couple dozen so far.

[poweradminllc]

That is quite surprising. As you know from the code PAExec just uses standard Windows APIs, and I can't think of why a subnet would matter.

Try taking PAExec out of the picture and just uses services.msc. Can you connect to foo.contoso.com and start/stop services or do you get access denied there as well?

[ghost]

I was able to connect Services.msc to one of the problem computers and restart the WinRM service without any issue. I can also get to the ADMIN$ share just fine.

[poweradminllc]

You didn't need to give any credentials to services.msc when connecting to foo.contoso.com? That would mean you're logged in as a domain admin account to which foo.contso.com also belongs right?

[ghost]

Ooohhh, I think I found something interesting. Looking at the Windows Events on the failed computer, PAExec service is successfully installed by the Admin account credentials that were provided, however the service fails to start because it is trying to use the standard user that is currently logged into the system to start the service.

Looking at PSExec, it registers the PSEXESVC service and it runs under LocalSystem (just like PAExec*), so something weird is happening with the start service section. What's more, this is happening even on a machine where the locally logged in user is supposed to be an administrator.

You didn't need to give any credentials to services.msc when connecting to foo.contoso.com? That would mean you're logged in as a domain admin account to which foo.contso.com also belongs right?

I'm Running As a domain account with admin rights, yes, but I am not logged-in as one. I get the same errors if I try to use the -u admin@contoso.com switch and give the password when prompted.

[poweradminllc]

Wow. That's surprising. It probably explains the problem, but I don't understand how. If you look in Remote.cpp at the InstallAndStartRemoteService function, the service is created, and then started, all either running as the current user, or all as an impersonated user. There's no impersonation changes between creation and starting.

[ghost]

I assume that hSCM is what contains the credential information, and that is eventually included in hService when Creating and Starting the service after making a successful connection? The Impersonation bit looks weird to me, but I'm not well-versed in C++ so it all looks weird to me. lol

I think I'll clone the repo at some point, add a bunch of log output and follow it step by step and see if I find anything useful. That's going to have to wait for another day though. At least we know that there's something weird happening with the logins.

[ghost]

So I started testing PAExec against localhost and running as System to see if things were any different, and I finally found the issue. For whatever reason, our AV software was blocking it on most of our computers, but not all of them. When went to run locally on a machine that wasn't working remotely, I received a popup notification of PAExec being blocked as a PUA; so I'm running it by SOC to get it whitelisted.

Sorry for the false-alarm.

I did find one interesting tidbit though. According to Audit logs in Windows Events, PAExec seems to open 2 logon session ID's, but only terminates one logon session. (PSExec only creates 1 logon session) Since the service gets deleted, I assume the second session is auto-closed and therefore not a security concern. My guess is OpenSCManager, EstablishConnection, or both are creating these sessions; maybe there's something to improve where only one logon session is necessary?

[poweradminllc]

Thanks for the update. I think we found in the code where that other logon session is coming from and are making a change to address it.