strimzi / strimzi-kafka-operator

Apache Kafka® running on Kubernetes
https://strimzi.io/
Apache License 2.0
4.76k stars 1.27k forks source link

[Enhancement]: Allow configmaps/secrets to be used for javaSystemProperties #9967

Closed CameronHudson8 closed 5 months ago

CameronHudson8 commented 5 months ago

Related problem

I'm trying to set up a Strimzi KafkaConnect custom resource with a KafkaConnector based on the MongoDB Kafka Connector.

I'm able to get it to work with MongoDB X509 authentication, but to do so, I need to hardcode the passwords of the Java trust store and key store files in the KafkaConnect manifest. Example:

# Working

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  annotations:
    strimzi.io/use-connector-resources: "true"
  name: my-mongo
  namespace: my-namespace
spec:
  externalConfiguration:
    volumes:
      - name: mongo-connection
        configMap:
          name: mongo-connection
      - name: mongo-creds
        secret:
          secretName: mongo-creds
  jvmOptions:
    # Inspired by https://www.mongodb.com/docs/kafka-connector/current/security-and-authentication/tls-and-x509/#add-credentials-to-the-connector
    javaSystemProperties:
      - name: javax.net.ssl.trustStore
        value: /opt/kafka/external-configuration/mongo-connection/ca-cert.p12
      # Not ideal
      - name: javax.net.ssl.trustStorePassword
        value: 'KYmpi^aUvkA7#kFp'
      - name: javax.net.ssl.keyStore
        value: /opt/kafka/external-configuration/mongo-creds/client-cert-and-key.p12
      # Not ideal
      - name: javax.net.ssl.keyStorePassword
        value: 'R3Asf*E@m2Ddr3A7'

---

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  labels:
    strimzi.io/cluster: my-mongo
  name: my-mongo
  namespace: my-namespace
spec:
  class: com.mongodb.kafka.connect.MongoSourceConnector
  config:
    # Refer to https://www.mongodb.com/docs/drivers/java/sync/current/fundamentals/auth/#std-label-x509-auth-mechanism
    connection.uri: mongodb://my-host:27017&authMechanism=MONGODB-X509&tls=true

Suggested solution

Would it be possible to allow configmaps or secrets to be referenced here, similar to what's possible for the externalConfiguration property? This is an example of what I'm imagining, inspired by the EnvVar kubernetes property:

kind: KafkaConnect
spec:
  jvmOptions:
    # Inspired by https://www.mongodb.com/docs/kafka-connector/current/security-and-authentication/tls-and-x509/#add-credentials-to-the-connector
    javaSystemProperties:
      - name: javax.net.ssl.trustStore
        value: /opt/kafka/external-configuration/mongo-connection/ca-cert.p12
      - name: javax.net.ssl.trustStorePassword
        valueFrom:
          configMapKeyRef:
            name: mongo-connection
            key: ca-cert.p12.password
      - name: javax.net.ssl.keyStore
        value: /opt/kafka/external-configuration/mongo-creds/client-cert-and-key.p12
      - name: javax.net.ssl.keyStorePassword
        valueFrom:
          secretKeyRef:
            name: mongo-creds
            key: client-cert-and-key.p12.password

Alternatives

No response

Additional context

No response

scholzj commented 5 months ago

This has been discussed many times in the past. Overriding the default truststore / keystore is a bad security practice. You should not do it. The MongoDB connector should do what most other connectors do and provide proper configuration options to specify its trust store and its password.

scholzj commented 5 months ago

FYI: I guess these are the options you should use: https://jira.mongodb.org/browse/KAFKA-348

scholzj commented 5 months ago

Discussed on the community call on 18.4.: As explained above, the connector configuration options should be used instead of changing the default stores. This should be closed.