search

ibmruntimes / openj9-openjdk-jdk8

Extensions for OpenJDK8 for Eclipse OpenJ9
GNU General Public License v2.0
52 stars 113 forks source link

Update the OpenJDK exclude tests list for FIPS #611

Open WilburZjh opened 1 year ago

WilburZjh commented 1 year ago

The FIPS mode has restrictive security configurations, so update the OpenJDK exclude tests list “ProblemList-fips.txt” for the FIPS testing.

WilburZjh commented 1 year ago

Exclude tests list from sanity.openjdk

java.security.NoSuchProviderException: no such provider: SunRsaSign

Existing Jars sign related

-keystore ks -storepass changeit -keypass changeit -keyalg rsa -alias a -dname CN=A -genkeypair keytool error: java.security.KeyStoreException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID

WilburZjh commented 1 year ago

Exclude tests list from extended.openjdk

java.security.KeyStoreException: JCEKS not found

java.security.KeyStoreException: JKS not found java.security.NoSuchAlgorithmException: JKS KeyStore not available

no such algorithm: DSA for provider SUN

Hard code provider SunJCE

no such provider: SunJCE

no such provider: SunRsaSign

java.lang.RuntimeException: Provider SunRsaSign not found.

Hard code provider SunRsaSign

SunJGSS provider & krb5 related

XMLDSig related DOM XMLSignatureFactory not available DOM mechanism not available DOM KeyInfoFactory not available

failed to load PKCS12 keystore related java.io.IOException: parseAlgParameters failed: PBE AlgorithmParameters not available at sun.security.pkcs12.PKCS12KeyStore.parseAlgParameters(PKCS12KeyStore.java:828) at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2047) at java.security.KeyStore.load(KeyStore.java:1445) at Utils.loadKeyStore(Utils.java:125)
Caused by: java.security.NoSuchAlgorithmException: PBE AlgorithmParameters not available at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) at java.security.Security.getImpl(Security.java:714) at java.security.AlgorithmParameters.getInstance(AlgorithmParameters.java:146) at sun.security.pkcs12.PKCS12KeyStore.parseAlgParameters(PKCS12KeyStore.java:822)

Caused by: java.security.NoSuchAlgorithmException: PBES2 AlgorithmParameters not available

FIPS doesnt support DES secret key

check if a provider exactly equals to Sun

no such algorithm: MD2 for provider SUN

no such algorithm: MD5 for provider SUN

no such algorithm: NONEwithDSA for provider SUN

no such algorithm: PKCS11 for provider SUN

no such algorithm: SHA for provider SUN

no such algorithm: SHA-256 for provider SUN

NativePRNG SecureRandom not available

SHA1PRNG SecureRandom not available

SHA-512/224 MessageDigest not available

SHA-512/256 MessageDigest not available

DSS Signature not available

SunPKCS11-Solaris provider related

unrecognized algorithm name: PBKDF2WITHHMACSHA1

unrecognized algorithm name: PBEWITHMD5ANDDES

RSASSA-PSS AlgorithmParameters not available

failed to save PKCS12 keystore related. PBEWithSHA1AndRC2_40 AlgorithmParameters not available

load PBE SecretKeyFactory not available

RSASSA-PSS KeyPairGenerator not available

failed to translate a DSA public key from P11DSAKeyFactory java.lang.NullPointerException at sun.security.pkcs11.P11DSAKeyFactory.implTranslatePublicKey(P11DSAKeyFactory.java:57) at sun.security.pkcs11.P11DSAKeyFactory.engineGeneratePublic(P11DSAKeyFactory.java:109) at java.security.KeyFactory.generatePublic(KeyFactory.java:334)

failed to translate a DSA private key from P11DSAKeyFactory java.lang.NullPointerException at sun.security.pkcs11.P11DSAKeyFactory.implTranslatePrivateKey(P11DSAKeyFactory.java:83) at sun.security.pkcs11.P11DSAKeyFactory.engineGeneratePrivate(P11DSAKeyFactory.java:141) at java.security.KeyFactory.generatePrivate(KeyFactory.java:372)

Failed to translate a DSA private key get from KeyPairGenerator

Could not create RSA private key due to the CKA_CLASS is CKO_PRIVATE_KEY Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID at sun.security.pkcs11.wrapper.PKCS11$InnerPKCS11.C_CreateObject(PKCS11.java:183) at sun.security.pkcs11.P11RSAKeyFactory.generatePrivate(P11RSAKeyFactory.java:250) at sun.security.pkcs11.P11RSAKeyFactory.implTranslatePrivateKey(P11RSAKeyFactory.java:80) Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID at sun.security.pkcs11.wrapper.PKCS11$InnerPKCS11.C_CreateObject(PKCS11.java:183) at sun.security.pkcs11.P11RSAKeyFactory.generatePrivate(P11RSAKeyFactory.java:250) at sun.security.pkcs11.P11RSAKeyFactory.implTranslatePrivateKey(P11RSAKeyFactory.java:80)

sun.security.pkcs11.P11Key$P11PrivateKey incompatible with sun.security.provider.DSAPrivateKey

Cant create a PKCS12 keystore related

Cannot find any provider supporting AESWrap

DES/DESede SecretKey algorithms is not supported

PBEWithSHA1AndRC2_40 AlgorithmParameters not available

PBEWithMD5AndDES SecretKeyFactory not available

PBEWithHmacSHA1AndAES_128 SecretKeyFactory not available

PBKDF2WithHmacSHA1/PBKDF2WithHmacSHA224/PBKDF2WithHmacSHA256/PBKDF2WithHmacSHA384/PBKDF2WithHmacSHA512 SecretKeyFactory not available

pbeWithMD5ANDdes SecretKeyFactory not available

PBEWithSHA1AndDESede AlgorithmParameters not available

FIPS does not support DES/ECB/PKCS5Padding

Cannot find any provider supporting PBEWithMD5AndTripleDES

Cannot find any provider supporting Blowfish

unrecognized algorithm name: PBEWithMD5AndDES

P11KeyPairGenerator doesnt support DesEDE

Could not create DH private key

sun.security.pkcs11.P11Key$P11DSAPublicKey incompatible with sun.security.x509.X509Key

XMLSignature provider is disabled in FIPS mode

Module java.security.sasl related. Unable to find client impl for CRAM-MD5 or DIGEST-MD5. NTLM should not support auth-conf

java.security.ProviderException: Could not derive key at sun.security.pkcs11.P11ECDHKeyAgreement.engineGenerateSecret(P11ECDHKeyAgreement.java:145) at javax.crypto.KeyAgreement.generateSecret(KeyAgreement.java:586)

java.security.spec.InvalidKeySpecException: Could not create EC private key at sun.security.pkcs11.P11ECKeyFactory.engineGeneratePrivate(P11ECKeyFactory.java:218) at java.security.KeyFactory.generatePrivate(KeyFactory.java:366) at TestECDH2.genECKeyPair(TestECDH2.java:98)

java.security.KeyStoreException: Key protection algorithm not found: java.lang.NullPointerException at sun.security.pkcs12.PKCS12KeyStore.setKeyEntry(PKCS12KeyStore.java:687) at sun.security.pkcs12.PKCS12KeyStore.engineSetEntry(PKCS12KeyStore.java:1408) at java.security.KeyStore.setEntry(KeyStore.java:1557)

java.security.spec.InvalidKeySpecException: Could not create DH private key at sun.security.pkcs11.P11DHKeyFactory.engineGeneratePrivate(P11DHKeyFactory.java:166) at java.security.KeyFactory.generatePrivate(KeyFactory.java:366) Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID at sun.security.pkcs11.wrapper.PKCS11.C_CreateObject(Native Method) at sun.security.pkcs11.wrapper.PKCS11$InnerPKCS11.C_CreateObject(PKCS11.java:183) at sun.security.pkcs11.P11DHKeyFactory.generatePrivate(P11DHKeyFactory.java:207) at sun.security.pkcs11.P11DHKeyFactory.engineGeneratePrivate(P11DHKeyFactory.java:160)

Caused by: java.security.ProviderException: Initialization failed at sun.security.pkcs11.SunPKCS11.(SunPKCS11.java:421) at sun.security.pkcs11.SunPKCS11.(SunPKCS11.java:125) Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_SLOT_ID_INVALID at sun.security.pkcs11.SunPKCS11.(SunPKCS11.java:387) Initialization failed PKCS11Exception: CKR_SLOT_ID_INVALID. All the below tests will call PKCS11Test.getSunPKCS11(PKCS11Test.java:105) to get the SunPKCS11 provider. When testing in the FIPS mode, the SunPKCS11 will first be initialized as a FIPS provider SunPKCS11-NSS-FIPS. And then in the test code PKCS11Test, line 480 and 486. It will try to configure the SunPKCS11 using the p11-nss.txt to the NSS mode. But in the FIPS mode, there can only be a single PKCS11 provider. So configure the SunPKCS11 to the NSS mode will failed.

java.lang.IllegalArgumentException: if keyStoreType is PKCS11, then keyStore must be NONE

javax.security.auth.login.LoginException: if keyStoreType is PKCS11 then keyStoreURL must be NONE

Could not generate keyPair for DH java.security.ProviderException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ARGUMENTS_BAD at sun.security.pkcs11.P11KeyPairGenerator.generateKeyPair(P11KeyPairGenerator.java:424) at java.security.KeyPairGenerator$Delegate.generateKeyPair(KeyPairGenerator.java:697)

All the below hard coded static String keyStoreFile = "keystore", and set the system property; However, in the test codes. In FIPS mode, keystore must be NONE. javax.net.ssl.SSLHandshakeException: no cipher suites in common

NSS module initial failures. It using "nss.cfg"/"nsstrust.cfg"/"nsscrypto.cfg" as the configure file and in the FIPS mode, there can only be a single PKCS11 provider. java.security.ProviderException: NSS library directory /usr/lib64 invalid, NSS already initialized with /usr/lib64/

FIPS doesnt support ECB mode in AES/ECB/PKCS5Padding cipher java.security.InvalidKeyException: Could not create key at sun.security.pkcs11.P11SecretKeyFactory.createKey(P11SecretKeyFactory.java:274) at sun.security.pkcs11.P11SecretKeyFactory.convertKey(P11SecretKeyFactory.java:179) at sun.security.pkcs11.P11SecretKeyFactory.convertKey(P11SecretKeyFactory.java:111)

com.sun.exp.provider.EXP related

FIPS doesnt support DES secret key

Failed to create a PKCS12 keystore

FIPS doesnt support DESede algorithms

Cipher suites mismatch

FIPS doesnt support DKS keystore

Policy file related. Failed due to related to the keystore files.

Hard code checking if the provider is "MyProvider()"

keytool error: java.security.KeyStoreException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID

Cant create a keystore in FIPS by sun.security.tools.keytool.Main.main( ("-keystore jks -storepass changeit -keypass changeit " + "-dname CN=A -alias " + keyalg + " -genkeypair " + "-keyalg " + keyalg).split(" ")); or -keystore ks -storepass changeit -keypass changeit

We can generate different such as SslMacMD5/SslMacSHA1/HmacMD5/HmacSHA1/HmacSHA224/HmacSHA256/HmacSHA384/HmacSHA512 secret keys but we only support to import AES secret key for now. java.security.InvalidKeyException: init() failed at sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:208) Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_TYPE_INCONSISTENT at sun.security.pkcs11.wrapper.PKCS11.C_SignInit(Native Method) at sun.security.pkcs11.P11Mac.initialize(P11Mac.java:177) at sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:206)

Could not import DSA private key when invoke kpg.initialize(1024) after KeyPairGenerator kpg = KeyPairGenerator.getInstance("DSA");

We can generate DES secret keys but cant import it

testMDChange failed at:MD2/2500 This is failed because the BUFFER_SIZE is defined as 96 in P11Digest engine class for FIPS. The length (2500) of the data which is over the buffer size.

Exception in thread "main" java.security.ProviderException: Unknown mechanism: 20 mechanism 20 is: SHA256withDSA, its alias is CKM_DSA_SHA256. It is defined in SunPKCS11 when fips mode is enabled. When getInstance("SHA256withDSA") in invoked from Signature class, it will eventually invoke the P11Signature(token, algorithm, mechanism). However, in the P11Signature constructor, there is no CKM_DSA_SHA256 defined in the switch-case statement. Not only CKM_DSA_SHA256, but also CKM_DSA_SHA224/CKM_DSA_SHA384/CKM_DSA_SHA512, etc. Thereofore, SHA256withDSA is an Unknown mechanism.

Can generate a TlsMasterSecret key via (TlsMasterSecret)kg.generateKey() but the result of key.getEncoded() is null; It seems that cant import a TlsMasterSecret key;

When FIPS mode is disabled, cacerts in ${JAVA_HOME}/jre/lib/security/cacerts is not empty. Therefore, X509Certificate[] acceptedIssuers = trustManager.getAcceptedIssuers(); The length of acceptedIssuers is not 0. But when FIPS mode is enabled, cacerts in ${JAVA_HOME}/jre/lib/security/cacerts is empty. While executing this statement X509Certificate[] acceptedIssuers = trustManager.getAcceptedIssuers(); The length of acceptedIssuers is 0.

WilburZjh commented 1 year ago

detail explanation for #612

sshuklao commented 1 year ago

I am getting same CKR_KEY_TYPE_INCONSISTENT error when running Apache Spark 3.3.0 on IBM-Semeru-Runtime 11.0.16.1 on FIPS enabled cluster.

Caused by: java.security.InvalidKeyException: init() failed
    at jdk.crypto.cryptoki/sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:208)
    at java.base/javax.crypto.Mac.chooseProvider(Mac.java:366)
    at java.base/javax.crypto.Mac.init(Mac.java:435)
    at com.ibm.stocator.thirdparty.cos.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:127)
    ... 34 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_TYPE_INCONSISTENT
    at jdk.crypto.cryptoki/sun.security.pkcs11.wrapper.PKCS11.C_SignInit(Native Method)
    at jdk.crypto.cryptoki/sun.security.pkcs11.P11Mac.initialize(P11Mac.java:177)
    at jdk.crypto.cryptoki/sun.security.pkcs11.P11Mac.engineInit(P11Mac.java:206)
    ... 37 more
WilburZjh commented 1 year ago

The traceStack for javax/crypto/CipherSpi/TestGCMWithByteBuffer.java is 

javax.crypto.BadPaddingException: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ENCRYPTED_DATA_INVALID
    at sun.security.pkcs11.P11AEADCipher.handleException(P11AEADCipher.java:749)
    at sun.security.pkcs11.P11AEADCipher.implDoFinal(P11AEADCipher.java:722)
    at sun.security.pkcs11.P11AEADCipher.engineDoFinal(P11AEADCipher.java:548)
    at javax.crypto.Cipher.doFinal(Cipher.java:2463)
    at TestGCMWithByteBuffer.decrypt(TestGCMWithByteBuffer.java:157)
    at TestGCMWithByteBuffer.test(TestGCMWithByteBuffer.java:86)
    at TestGCMWithByteBuffer.main(TestGCMWithByteBuffer.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:298)
    at java.lang.Thread.run(Thread.java:826)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ENCRYPTED_DATA_INVALID
    at sun.security.pkcs11.wrapper.PKCS11.C_Decrypt(Native Method)
    at sun.security.pkcs11.P11AEADCipher.implDoFinal(P11AEADCipher.java:708)

In NSS docs version 3.15, it is mentioned that for Cipher mode AES-GCM.

PK11_Decrypt - Performs decryption as a single PKCS#11 operation (eg: not multi-part). This is necessary for AES-GCM.
PK11_Encrypt - Performs encryption as a single PKCS#11 operation (eg: not multi-part). This is necessary for AES-GCM.
WilburZjh commented 1 year ago

The traceStack for sun/rmi/rmic/manifestClassPath/run.sh is

Exception in thread "main" java.lang.ExceptionInInitializerError
    at java.lang.J9VMInternals.ensureError(J9VMInternals.java:158)
    at java.lang.J9VMInternals.recordInitializationFailure(J9VMInternals.java:147)
    at sun.rmi.transport.LiveRef.<init>(LiveRef.java:74)
    at sun.rmi.server.UnicastServerRef.<init>(UnicastServerRef.java:168)
    at sun.rmi.server.UnicastServerRef.<init>(UnicastServerRef.java:186)
    at java.rmi.server.UnicastRemoteObject.exportObject(UnicastRemoteObject.java:301)
    at Main.main(Main.java:10)
Caused by: java.lang.RuntimeException: could not find SunPKCS11-NSS-FIPS provider for FIPS mode
    at java.security.SecureRandom.getDefaultPRNG(SecureRandom.java:209)
    at java.security.SecureRandom.<init>(SecureRandom.java:170)
    at java.rmi.server.UID.<init>(UID.java:112)
    at java.rmi.server.ObjID.<clinit>(ObjID.java:88)
    ... 5 more

If we assign a directory to -Djava.ext.dirs jvm options, the ExtClassLoader will not load the jar file under ${JAVA_HOME}/jre/lib/ext.

One solution that fix this issue is expanding -Djava.ext.dirs with {JAVA_HOME}/jre/lib/ext. A passed version can be seen from this <<grinder>>

WilburZjh commented 1 year ago

The exception is : sun.security.pkcs11.wrapper.PKCS11Exception: CKR_KEY_SIZE_RANGE. The minimum length for an MD5 HMAC key is 16 bytes. However, this test uses a 4 bytes length key for HmacMD5.