jborean93 / omi

Open Management Infrastructure
Other
109 stars 13 forks source link

docker ubuntu aarch64 "acquiring creds with username only failed An invalid name was supplied SPNEGO cannot find mechanisms to negotiate" #71

Open carefreepineapple opened 9 months ago

carefreepineapple commented 9 months ago
SUMMARY

When trying to Enter a PSSession, it fails

PS /> Enter-PSSession -ComputerName "<redacted>" -Credential $creds -Authentication Negotiate -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck)
Enter-PSSession: Connecting to remote server <redacted> failed with the following error message : acquiring creds with username only failed An invalid name was supplied SPNEGO cannot find mechanisms to negotiate For more information, see the about_Remote_Troubleshooting Help topic.

As this seems like an NTLM issue, I've made sure I installed the gss-ntlmssp package, but it makes no difference. Hoping someone can point me in the right direction.

MODULE VERSION
Not applicable as `libmi.so` and `libpsrpclient.so` were manually built.
OS / ENVIRONMENT

Ubuntu 22.04docker container on Mac M2 (ARM)

root@611f98637ea0:/# cat etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
root@611f98637ea0:/# uname -a
Linux 611f98637ea0 6.5.11-linuxkit #1 SMP PREEMPT Wed Dec  6 17:08:31 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

Powershell Version

PS /> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.1.5
PSEdition                      Core
GitCommitId                    7.1.5
OS                             Linux 6.5.11-linuxkit #1 SMP PREEMPT Wed Dec  6 17:08:31 UTC 2023
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Libraries

root@611f98637ea0:/# ls -la /opt/microsoft/powershell/7/libmi.so /opt/microsoft/powershell/7/libpsrpclient.so
lrwxrwxrwx 1 root root 39 Feb  6 11:56 /opt/microsoft/powershell/7/libmi.so -> /opt/omi/build/lib/ubuntu22.04/libmi.so
lrwxrwxrwx 1 root root 47 Feb  6 11:56 /opt/microsoft/powershell/7/libpsrpclient.so -> /opt/omi/build/lib/ubuntu22.04/libpsrpclient.so
STEPS TO REPRODUCE

Here is the very rough dockerfile i'm using to create the image

FROM ubuntu:latest
LABEL maintainer="carefreepineapple"

SHELL ["/bin/bash", "-c"]

RUN apt update -y
RUN DEBIAN_FRONTEND=noninteractive apt upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt install -y iputils-ping ansible telnet vim tcpdump git python-is-python3
RUN DEBIAN_FRONTEND=noninteractive apt install -y wget apt-transport-https software-properties-common sshpass gss-ntlmssp
RUN source /etc/os-release && wget -c -P /tmp -q https://packages.microsoft.com/config/ubuntu/$VERSION_ID/packages-microsoft-prod.deb
RUN DEBIAN_FRONTEND=noninteractive apt install -y libc6 libgcc1 libgcc-s1 libgssapi-krb5-2 libicu70 liblttng-ust1 libssl3 libstdc++6 libunwind8 zlib1g
RUN wget -c -P /tmp -q http://ports.ubuntu.com/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2_arm64.deb
RUN dpkg -i /tmp/libssl1.1_1.1.1f-1ubuntu2_arm64.deb
RUN mkdir -p /opt/microsoft/powershell/7
RUN wget -c -P /tmp -q https://github.com/PowerShell/PowerShell/releases/download/v7.1.5/powershell-7.1.5-linux-arm64.tar.gz
RUN tar -xvzf /tmp/powershell-7.1.5-linux-arm64.tar.gz -C /opt/microsoft/powershell/7/
RUN ln -s /opt/microsoft/powershell/7/pwsh /usr/local/bin
RUN git clone https://github.com/jborean93/omi.git /opt/omi
RUN DEBIAN_FRONTEND=noninteractive apt install -y cmake git libkrb5-dev libssl-dev make openssl pkg-config
RUN cd /opt/omi/Unix && /opt/omi/Unix/configure --outputdirname="build-ubuntu22.04" --prefix="/opt/omi" && make -j
RUN mkdir -p /opt/omi/build/lib/ubuntu22.04 && cp /opt/omi/Unix/build-ubuntu22.04/lib/libmi.so /opt/omi/build/lib/ubuntu22.04/
RUN git clone https://github.com/PowerShell/psl-omi-provider.git /opt/omi/psl-omi-provider/repo
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/1.BuildFixes.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/2.AuthenticateDefault.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/3.FixUninitializedVar.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/4.VersionInfo.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/5.CertificateCheck.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/6.NoChkshlibBuild.diff"
RUN cd /opt/omi/psl-omi-provider/repo && git apply "/opt/omi/psl-omi-provider/7.CMakeLanguage.diff"
RUN rm -rf /opt/omi/psl-omi-provider/repo/omi
RUN ln -s /opt/omi /opt/omi/psl-omi-provider/repo/omi
RUN ln -s /opt/omi/Unix/build-ubuntu22.04/ /opt/omi/Unix/output
RUN cd /opt/omi/psl-omi-provider/repo/src && cmake -DCMAKE_BUILD_TYPE='Release' . && make psrpclient
RUN cp /opt/omi/psl-omi-provider/repo/src/libpsrpclient.* /opt/omi/build/lib/ubuntu22.04/
RUN ldd /opt/omi/build/lib/ubuntu22.04/libpsrpclient.so || true
RUN ldd /opt/omi/build/lib/ubuntu22.04/libmi.so || true
RUN ln -s /opt/omi/build/lib/ubuntu22.04/libmi.so /opt/microsoft/powershell/7/
RUN ln -s /opt/omi/build/lib/ubuntu22.04/libpsrpclient.so /opt/microsoft/powershell/7/
CMD ["/bin/bash"]
jborean93 commented 9 months ago

Did you enable gss-ntlmssp in mechs config file, I believe Debian based distros place it under /etc/gss/mech.d/ but I could be wrong there. If you haven't already try using the UPN format for the username user@REALM.COM instead of REALM\user.

carefreepineapple commented 9 months ago

@jborean93 yes, i am seeing gss-ntlmssp with a mech.ntmssp.conf file

root@611f98637ea0:/etc/gss/mech.d# cat mech.ntlmssp.conf
# NTLMSSP mechanism plugin
#
# NOTE: to activate the NTLM SSP mechanism do the following
#     * FOR krb5 < 1.12: copy the content of this file in /etc/gss/mech
#     * FOR krb5 >= 1.12: copy this file in /etc/gss/mech.d
#
# Mechanism Name    Object Identifier       Shared Library Path         Other Options
gssntlmssp_v1       1.3.6.1.4.1.311.2.2.10          /usr/lib/aarch64-linux-gnu/gssntlmssp/gssntlmssp.so

root@611f98637ea0:/etc/gss/mech.d# ls -la /usr/lib/aarch64-linux-gnu/gssntlmssp/gssntlmssp.so
-rw-r--r-- 1 root root 100968 Dec  1  2021 /usr/lib/aarch64-linux-gnu/gssntlmssp/gssntlmssp.so

The windows host is not on a domain, so i've tried the username and username@domain.local, neither change the behavior.

jborean93 commented 9 months ago

Unfortunately I'm not sure what the problem here is without stepping through the code. You could enable debug logging under https://github.com/jborean93/omi?tab=readme-ov-file#troubleshooting but I don't think this shows authentication details.

For whatever reason the call to gss_acquire_cred_with_password is failing. If you have Python you could try and do the following to test out a manual call to this function:

# podman run --rm -it ubuntu:22.04

apt-get update
apt-get install -y gcc python3 python3-dev python3-pip libkrb5-dev gss-ntlmssp
python3 -m pip install gssapi

python3 <<EOF
import gssapi

username = 'username'
password = b'password'
target = 'target-service'

mechs = [
    gssapi.OID.from_int_seq('1.3.6.1.5.5.2'),  # Spnego
    gssapi.OID.from_int_seq('1.2.840.113554.1.2.2'),  # Kerberos
    gssapi.OID.from_int_seq('1.3.6.1.4.1.311.2.2.10'),  # NTLM
]
cred = gssapi.raw.acquire_cred_with_password(
    gssapi.Name('username', name_type=gssapi.NameType.user),
    b'password',
    usage='initiate',
    mechs=mechs).creds

ctx = gssapi.SecurityContext(
    name=gssapi.Name(f'http@{target}', name_type=gssapi.NameType.hostbased_service),
    creds=cred,
    mech=mechs[0],
    usage='initiate')
print(ctx.step())
EOF

You should see roughly in the middle a value NTLMSSP\x00\x01 which means it was able to produce a valid NTLM token. As it's a local user then the username by itself should be fine.

When running the above in an Ubuntu 22.04 container I get

Traceback (most recent call last):
  File "<stdin>", line 12, in <module>
  File "gssapi/raw/ext_password.pyx", line 75, in gssapi.raw.ext_password.acquire_cred_with_password
gssapi.raw.misc.GSSError: Major (851968): Unspecified GSS failure.  Minor code may provide more information, Minor (1314127875): Crypto routine failure

This would indicate the crypto policies is disabling algorithms required by NTLM (like md4, rc4, etc). You can re-enable them by allowing the legacy providers in OpenSSL either in the global file or through a custom config. For example with the above and a custom policy file I get

cat > /tmp/openssl-legacy.cnf <<EOF
openssl_conf = openssl_init

[openssl_init]
providers = provider_sect

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1
EOF

export OPENSSL_CONF=/tmp/openssl-legacy.cnf
python3 ...

> b'`@\x06\x06+\x06\x01\x05\x05\x02\xa0604\xa0\x0e0\x0c\x06\n+\x06\x01\x04\x01\x827\x02\x02\n\xa2"\x04 NTLMSSP\x00\x01\x00\x00\x00\x17\x82\x08\xa0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
carefreepineapple commented 9 months ago

@jborean93 thank you for your help. I tried your python script and it worked successfully, so know we can product a valid NTLM token.

So after updating /etc/ssl/openssl.cnf with the following updates:

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1

Now when I try Enter-PSSession, I get this error:

PS /> Enter-PSSession -ComputerName "<redacted>" -Credential $creds -Authentication Negotiate -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck)
Enter-PSSession: Connecting to remote server <redacted> failed with the following error message : MI_RESULT_FAILED For more information, see the about_Remote_Troubleshooting Help topic.

And we're onto MI_RESULT_FAILED, which is progress!

I added the logging conf for omi (DEBUG) and got this:

Server: Microsoft-HTTPAPI/2.0
Date: Wed, 07 Feb 2024 04:16:04 GMT
Connection: close
Content-Length: 0

ed, 07 Feb 2024 04:16:04 GMT
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadHeader - Checking for full header...
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadHeader - Full header has been received
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadHeader - OK exit
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadData - Begin. Head? 0
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadData - Received size: 0 / 0
2024/02/07 04:16:07 [145,170] DEBUG: null(0): HttpClient_DecryptData - No content type, not encrypted
2024/02/07 04:16:07 [145,170] DEBUG: null(0): _ReadData - HttpClient_DecryptData failure
2024/02/07 04:16:07 [145,170] DEBUG: null(0): EventId=45128 Priority=DEBUG InteractionProtocolHandler_Operation_Strand_Post: 0xfffee4175c30, msg(0xfffedc027638:4:PostResultMsg:0)
2024/02/07 04:16:07 [145,170] DEBUG: null(0): EventId=45129 Priority=DEBUG MI_Result = MI_RESULT_FAILED
2024/02/07 04:16:07 [145,170] DEBUG: null(0): EventId=45307 Priority=DEBUG MI_Client Operation Instance Result (async): session=0xfffee4019da8, operation=0xfffee4019dc0, internal-opera
tion=0xfffee4175570, resultCode=1, moreResults=FALSE

from what I can see, the function HttpClient_DecryptData is part of httpclientauth.c, but i'm not good with C. I know the credentials are good and they dont contain special characters, so that shouldn't be causing an issue.

Maybe I did something wrong when building the libraries?

In case its helpful, heres a decoded auth tokens:

root@7019e9be1fcc:/tmp# python3 -m spnego --format yaml --token <redacted>
MessageType: SPNEGO NegTokenResp
Data:
  negState: accept-incomplete (1)
  supportedMech: NTLM (1.3.6.1.4.1.311.2.2.10)
  responseToken:
    MessageType: CHALLENGE_MESSAGE (2)
    Data:
      TargetNameFields:
        Len: 22
        MaxLen: 22
        BufferOffset: 56
      NegotiateFlags:
        raw: 3800728117
        flags:
        - NTLMSSP_NEGOTIATE_56 (2147483648)
        - NTLMSSP_NEGOTIATE_KEY_EXCH (1073741824)
        - NTLMSSP_NEGOTIATE_128 (536870912)
        - NTLMSSP_NEGOTIATE_VERSION (33554432)
        - NTLMSSP_NEGOTIATE_TARGET_INFO (8388608)
        - NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY (524288)
        - NTLMSSP_TARGET_TYPE_SERVER (131072)
        - NTLMSSP_NEGOTIATE_ALWAYS_SIGN (32768)
        - NTLMSSP_NEGOTIATE_NTLM (512)
        - NTLMSSP_NEGOTIATE_SEAL (32)
        - NTLMSSP_NEGOTIATE_SIGN (16)
        - NTLMSSP_REQUEST_TARGET (4)
        - NTLMSSP_NEGOTIATE_UNICODE (1)
      ServerChallenge: A77D0C99FECD93FB
      Reserved: '0000000000000000'
      TargetInfoFields:
        Len: 120
        MaxLen: 120
        BufferOffset: 78
      Version:
        Major: 10
        Minor: 0
        Build: 17763
        Reserved: '000000'
        NTLMRevision: 15
      Payload:
        TargetName: <redacted>
        TargetInfo:
        - AvId: MSV_AV_NB_DOMAIN_NAME (2)
          Value: <redacted>
        - AvId: MSV_AV_NB_COMPUTER_NAME (1)
          Value: <redacted>
        - AvId: MSV_AV_DNS_DOMAIN_NAME (4)
          Value: <redacted>
        - AvId: MSV_AV_DNS_COMPUTER_NAME (3)
          Value: <redacted>
        - AvId: MSV_AV_TIMESTAMP (7)
          Value: '2024-02-07T04:16:04.6061061Z'
        - AvId: MSV_AV_EOL (0)
          Value:
    RawData:
      <redacted>
  mechListMIC:
RawData:
  <redacted>

second

root@7019e9be1fcc:/tmp# python3 -m spnego --format yaml --token <redacted>
MessageType: SPNEGO NegTokenResp
Data:
  negState: accept-complete (0)
  supportedMech:
  responseToken:
  mechListMIC: <redacted>
RawData: <redacted>
jborean93 commented 9 months ago

Is this still on the macOS or Ubuntu side? I can't say I'm aware of any encryption problems with anything outside of macOS but the code is quite messy so I can't guarantee there aren't any problems. Just as an FYI I tried out the docker example I shared above but with PSWSMan and it works.

# podman run --rm -it ubuntu:22.04

apt-get update
apt-get install -y apt-transport-https software-properties-common libkrb5-dev gss-ntlmssp wget
source /etc/os-release
wget -q https://packages.microsoft.com/config/ubuntu/$VERSION_ID/packages-microsoft-prod.deb
dpkg -i packages-microsoft-prod.deb
rm packages-microsoft-prod.deb

apt-get update
apt-get install -y powershell

pwsh -Command 'Install-Module -Name PSWSMan -Scope AllUsers -Force; Install-WSMan -Verbose'

cat > /tmp/openssl-legacy.cnf <<EOF
openssl_conf = openssl_init

[openssl_init]
providers = provider_sect

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1
EOF

OPENSSL_CONF=/tmp/openssl-legacy.cnf pwsh

$cred = Get-Credential HOSTNAME\user
Invoke-Command -ComputerName hostname { whoami } -Credential $cred

image

carefreepineapple commented 9 months ago

@jborean93 this is within an ubuntu 22.04 (aarch64) docker container thats running on a Mac OS m2 system.

I'll try what you have above, but I dont think PSWSMAN worked in the docker container last time I tried it.

EDIT: so since the container is run atop of the m2, I cant use the packages-microsoft-prod.deb as that only appears to be for x86. While it installs and gets added a repo, I'm unable to locate the powershell package. https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#unable-to-find-package

Using a package manager to install .NET from the Microsoft package feed only supports the x64 architecture. Other architectures, such as Arm, aren't supported by the Microsoft package feed.

EDIT 2: so I just learned you can run x86 containers on the ARM M2 platform. While shifting changes the original request, it should be sufficient for what I'm trying to do. Running an updated Dockerfile to try this and its pretty close to working, just the last command is failing (of course).

output:

root@323ae982d76f:/# pwsh -Command 'Install-WSMan'
Value cannot be null. (Parameter 'key')

root@323ae982d76f:/# pwsh
PowerShell 7.4.1
PS /> Install-WSMan

An error has occurred that was not properly handled. Additional information is shown below. The PowerShell process will exit.
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Regex1_Scan(RegexRunner, ReadOnlySpan`1)
Fatal error. Internal CLR error. (0x80131506)
   at DynamicClass.Regex1_Scan(System.Text.RegularExpressions.RegexRunner, System.ReadOnlySpan`1<Char>)
Aborted

Dockerfile:

#Image contains all the tools necessary to connect via powershell (winrm) to pwsh 5.1.

FROM ubuntu:latest
LABEL maintainer="carefreepineapple"

SHELL ["/bin/bash", "-c"]

RUN apt update -y
RUN DEBIAN_FRONTEND=noninteractive apt upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt install -y apt-transport-https software-properties-common libkrb5-dev gss-ntlmssp wget
RUN source /etc/os-release && wget -c -P /tmp -q https://packages.microsoft.com/config/ubuntu/$VERSION_ID/packages-microsoft-prod.deb
RUN dpkg -i /tmp/packages-microsoft-prod.deb
RUN apt update -y
RUN DEBIAN_FRONTEND=noninteractive apt install -y powershell
RUN pwsh -Command 'Install-Module -Name PSWSMan -Scope AllUsers -Force; Install-WSMan -Verbose'

CMD ["/bin/bash"]

this is then built with:

~$ docker buildx build --platform=linux/amd64 -t ubuntu-pwsh-test:latest -f ~/git/docker/ubuntu-pwsh/Dockerfile-pwsh-test .