How to set Bouncycastle as the security provider for Kafka broker?

[TheBestOrNothing]

To set security.providers do not work well for kafka broker and bouncycastle(BC) does not take effection. So we should make the bouncycastle(BC) with highest priority in the JVM.

security.providers=org.bouncycastle.jce.provider.BouncyCastleProvider,org.bouncycastle.jsse.provider.BouncyCastleJsseProvider

1. To find the java.security file with the command

cd /usr/lib/jvm/
find . -name java.security
vim ./java-17-openjdk-amd64/conf/security/java.security

2. Edit the java.security File. Here is the default configuration in Java 17

security.provider.1=SUN
security.provider.2=SunRsaSign
security.provider.3=SunEC
security.provider.4=SunJSSE
security.provider.5=SunJCE
security.provider.6=SunJGSS
security.provider.7=SunSASL
security.provider.8=XMLDSig
security.provider.9=SunPCSC
security.provider.10=JdkLDAP
security.provider.11=JdkSASL
security.provider.12=SunPKCS11

3. Add Bouncy Castle Provider: to make the bouncycastle(BC) with highest priority in the JVM

security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider
security.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider
security.provider.3=SUN
security.provider.4=SunRsaSign
security.provider.5=SunEC
security.provider.6=SunJSSE
security.provider.7=SunJCE
security.provider.8=SunJGSS
security.provider.9=SunSASL
security.provider.10=XMLDSig
security.provider.11=SunPCSC
security.provider.12=JdkLDAP
security.provider.13=JdkSASL
security.provider.14=SunPKCS11

4. At last, to reboot the system to make bouncycastle(BC) with highest priority take effection.

5. Restart Kafka Broker

[TheBestOrNothing]

After security.providers in jvm works, the broker should config as following. Then the kafka broker will use org.bouncycastle.jce.provider.BouncyCastleProvider as jce provider and use org.bouncycastle.jsse.provider.BouncyCastleJsseProvider as jsse provider.

security.providers=org.bouncycastle.jce.provider.BouncyCastleProvider,org.bouncycastle.jsse.provider.BouncyCastleJsseProvider

You can find detailed info from the configuration of kafka broker or server

[TheBestOrNothing]

The difference between JCE and JSSE from ChatGPT 3.5

In the context of Java, JCE (Java Cryptography Extension) and JSSE (Java Secure Socket Extension) are two different providers used for different purposes within the realm of cryptography and security.

JCE (Java Cryptography Extension):

Purpose: JCE is primarily focused on cryptographic operations and provides a framework for implementing cryptographic algorithms, such as encryption, decryption, hashing, and digital signatures, in Java applications. Common Use Cases: It is commonly used for securing data at rest, like encrypting files or sensitive data stored in databases. It can also be used for secure communication over non-network protocols. Examples of Usage: Encrypting and decrypting data using AES, RSA, or DES; creating and verifying digital signatures; calculating hash values; generating random numbers for cryptographic purposes. Providers: JCE provides a standard set of cryptographic algorithms and allows different cryptographic providers to be plugged in. Providers like "SunJCE" (the default provider in the Oracle JDK) offer implementations of these algorithms.

JSSE (Java Secure Socket Extension):

Purpose: JSSE is primarily focused on securing network communication using SSL/TLS protocols. It provides the necessary APIs and implementations for creating secure network connections, typically over TCP/IP. Common Use Cases: JSSE is commonly used in Java applications for securing communication between clients and servers, such as HTTPS (HTTP over TLS/SSL) for web applications, secure email communication (SMTPS, POP3S, IMAPS), and securing other network protocols. Examples of Usage: Creating secure socket connections, configuring SSL/TLS parameters, verifying server certificates, and establishing encrypted communication channels. Providers: JSSE also supports different cryptographic providers. "SunJSSE" is the default provider for SSL/TLS functionality in the Oracle JDK.

[TheBestOrNothing]

As to kafka client, we sholuld let bouncycastle JCE and JEES with highest poriority in the code.

        // Register BouncyCastleProvider when client (producer or consumer) start in main
        Security.insertProviderAt(new BouncyCastleProvider(), 1);
        Security.insertProviderAt(new BouncyCastleJsseProvider(), 2);

       //Set the client (producer and consumer) properties and configurations
      p.setProperty("security.providers",  
          "org.bouncycastle.jce.provider.BouncyCastleProvider,org.bouncycastle.jsse.provider.BouncyCastleJsseProvider");

You can find detailed info from the code of client (producer or consumer)