FIPS compliance with Java 11 (and later) on Windows

[colvinco]

Hello. We have a Java 11 application in Liberty, and would like to be able to configure Liberty in a FIPS 140-2 compliant manner on Windows Server 2016.

The documented support for FIPS compliance with Java 11 uses Semeru, and "support for FIPS with Semeru Runtimes is available only on Red Hat Enterprise Linux (RHEL) 8 on x86 platforms". I believe that IBM SDK 8 would have been an option on Windows if our application was still on Java 8, but Semeru is Linux/AIX only.

• So my first question is, is it fundamentally possible to configure Liberty with Java 11 on Windows in a FIPS compliant manner?
  • I think the answer to this is yes, it's just not documented.
  • As far as I understand it (but I may be wrong), compliance with FIPS 140-2 requires the use of TLS 1.0 or later with approved cryptographic modules and approved algorithms.
  • We already configure TLS 1.2
  • Liberty can be configured to use a specific list of ciphers, so the use of approved algorithms should be covered by that.
  • The use of approved modules depends on choosing an appropriate JVM / configuring it appropriately with approved security providers.
• My next question then is has anyone configured Liberty for FIPS with a Java 11 JRE other than Semeru (we are using Temurin) and can share some advice on how it was done?
  • I have tried configuring the Bouncy Castle FIPS module and it was being loaded for Liberty (confirmed with JProfiler, plus it throws an exception when reading LTPA keys if -Dorg.bouncycastle.fips.approved_only=true as the RSA key size for LTPA is only 1024 bits), however when I actually make a request to the application, the SSL handshake times out and doesn't give an obvious reason for not completing. (note: Without the BC FIPS module HTTPS is working normally)
  • I haven't dug into the BC documentation yet, so I probably haven't configured everything correctly, but it would good to know if anyone has successfully used BC FIPS (or another provider) with Liberty before I spend more time on it. This post from 5 years ago suggests that external providers aren't properly supported, but I see some changes have been made to JSSEProviderFactory since then.

Has anyone had any success with it? Thanks

---

For reference, I've got my java.security configured with

security.provider.1=org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
security.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider BCFIPS
security.provider.3=SUN
security.provider.4=SunRsaSign
security.provider.5=SunEC
security.provider.6=SunJSSE
security.provider.7=SunJCE

My jvm.options has

-Xbootclasspath/a:C:/<path_to_jar>/bc-fips-1.0.2.3.jar

If I include -Dorg.bouncycastle.fips.approved_only=true in it then I get

Caused by: org.bouncycastle.crypto.fips.FipsUnapprovedOperationError: Attempt to use RSA key with non-approved size: 1024: RSA
...
    at com.ibm.ws.crypto.ltpakeyutil.LTPACrypto.signISO9796(LTPACrypto.java:256)

So I know that BC is definitely being invoked.

[NottyCode]

We haven't tested Liberty with Bouncy Castle as the JSSE provider. From your report it sounds like it doesn't work.

FIPS 140-2 is being replaced with FIPS 140-3 so you can't get new certifications for it and the existing ones are starting to expire out. I'm not aware of a FIPS 140-3 certified JSSE either.

When we document FIPS we chose to focus on a working FIPS certified JSSE implementations which are on IBM Java 8, or for Seamer on Red Hat. You could possibly configure JSSE in Semeru to just allow the FIPS compatible options, but I don't know what those options would be.