OpenSSL error when making a connection from a FIPS-140 enabled Linux machine

adamk550 commented 7 months ago

We are getting failures making connections from a FIPS-140 enabled Linux machine (RHEL 9.2). We don't have the Kerberos or CredSSP support modules installed so it will be negotiating NTLM I think

This is the logging I have:

spnego._negotiate: DEBUG: SPNEGO step input: oYIBCzCCA...(redacted)
ERROR: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at
https://github.com/pyca/cryptography/issues
with information on how to reproduce this. ([_OpenSSLErrorWithText(code=50856204, lib=6, reason=524556, reason_text=b'error:0308010C:digital envelope routines::unsupported')])

My guess is that the spnego module is attempting to use something which is disabled in FIPS mode? The strange thing is that the same code works on a Windows machine with FIPS enabled .. but perhaps in that case its negotiating to use Kerberos and not NTLM

Not sure exactly how this work, if it should work at all in FIPS mode?

I don't have a standalone script which shows the problem but I can create one if that will help.

jborean93 commented 7 months ago

Is it possible to share how you are calling pypsrp and if there is a stacktrace that potentially shows where it is failing? If you are running with Ansible using -vvv might show that information.

adamk550 commented 7 months ago

I've written a simple script which shows what the code is doing and can be run standalone to show the issue. If run it on a non FIPS Linux VM:

==============================================================================
Python 3.11.4 (main, Jun 19 2023, 17:32:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4.3.0.1)]
Platform Linux-5.15.0-200.131.27.1.el9uek.x86_64-x86_64-with-glibc2.34
FIPS False
==============================================================================
INFO:pypsrp.powershell:Initialising RunspacePool object for configuration Microsoft.PowerShell
INFO:pypsrp.powershell:Opening a new Runspace Pool on remote host
INFO:pypsrp.powershell:Initialising PowerShell in remote Runspace Pool
INFO:pypsrp.powershell:Beginning remote Pipeline invocation
[{'PSVersion': '5.1.20348.859', 'PSEdition': 'Desktop', 'PSCompatibleVersions': ['1.0', '2.0', '3.0', '4.0', '5.0', '5.1.20348.859'], 'BuildVersion': '10.0.20348.859', 'CLRVersion': '4.0.30319.42000', 'WSManStackVersion': '3.0', 'PSRemotingProtocolVersion': '2.3', 'SerializationVersion': '1.1.0.1'}]
INFO:pypsrp.powershell:Closing Runspace Pool

While on a Linux VM with FIPS enabled:

==============================================================================
Python 3.11.4 (main, Jun 19 2023, 17:32:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4.3.0.1)]
Platform Linux-5.14.0-362.8.1.el9_3.x86_64-x86_64-with-glibc2.34
FIPS True
==============================================================================
INFO:pypsrp.powershell:Initialising RunspacePool object for configuration Microsoft.PowerShell
INFO:pypsrp.powershell:Opening a new Runspace Pool on remote host
... big long traceback ...

Looking at the traceback its in spnego/_ntlm_raw/crypto.py where is tries to create an RC4 cipher .. that won't work in FIPS mode. This page listed all the things you can't use in FIPS mode https://docs.oracle.com/cd/E36784_01/html/E54953/fips-notok-1.html (a useful Oracle document, who knew!)

I suspect that NTLM just won't work on a FIPS enabled machine as it seems to use RC4 and MD5, both of which are disabled in FIPS mode.

One further point, I tried the script on Windows and it worked in FIPS mode, even when I explicitly enabled FIPS and set auth to "ntlm" ... I don't understand that. In FIPS mode, RC4/MD5 etc should be blocked, that's kind of the point of FIPS! Perhaps on Windows its not using OpenSSL and the Windows crypto support behaves differently?

ps_fips.py.txt

jborean93 commented 7 months ago

Thanks for sharing the info, unfortunately NTLM is unfortunately fundamentally reliant on MD5 and RC4 during the authentication stage, if FIPS disables these then there is little we can really do about that. As for why it's working on Windows I can only really guess but I think what is happening is

The RC4 code is being called in the Python cryptography library
It contains a pre-built wheel for Windows that ships it's own copy of OpenSSL with it
This version of OpenSSL has no integration into Windows policies so is unaffected by the FIPS setting on Windows
This is unlike OpenSSL on other platforms where it could be reading a global openssl.cnf file that has enabled FIPS or the distro has installed their copy of the cryptography library that is linked to the system OpenSSL

At this point there is little I can do here. Theoretically I could implement both MD5 and RC4 in Python but I don't think it's a good idea to do so for speed purposes (RC4 is used for message wrapping and requires a non-Python solution) as well as FIPS compliance reasons.

adamk550 commented 7 months ago

Thanks for the feedback. I thought this would be the case.

As for Windows, you're right: I don't think my test system was actually in FIPS mode at all. When I've retested with our full setup the connection fails in the same way so its definitely the fact that these algorithms are disabled by FIPS.

I'll close this non issue

jborean93 / pypsrp

OpenSSL error when making a connection from a FIPS-140 enabled Linux machine #184