[Enabling one way TLS authentication in Kafka] ...

[vperi1730]

Hi Team,

I have a question regarding the enabling of the one way TLS authentication, How do we achieve this and what change do we need in the Kafka CR.

Let's say if I want to test producing messages on a particular topic on internal bootstrap with 9093, Do I need to disable the following configuration and then call the producer.sh by using only the Cluster CA crt, Is this called as 1-way TLS authentication?

tls:
        authentication:
          type: tls

./bin/kafka-console-producer.sh --broker-list mm-backup-cluster-kafka-bootstrap:9093 --topic mm-src-cluster.mm2-topic \
--producer-property security.protocol=SSL \
--producer-property ssl.truststore.type=PKCS12 \
--producer-property ssl.truststore.password=123456 \
--producer-property ssl.truststore.location=/tmp/certs/cluster.truststore.p12 

need your inputs and help here.

[scholzj]

Server authentication is basically just a regular TLS. So you can just remove the authentication section and keep something like tls: {}. That should work with the client config you have there.

For the record, this does not authenticate the client, so it will connect as anonymous user.

On Thu, Jul 2, 2020, 10:15 vperi1730 notifications@github.com wrote:

Hi Team,

I have a question regarding the enabling of the one way TLS authentication, How do we achieve this and what change do we need in the Kafka CR.

Let's say if I want to test producing messages on a particular topic on internal bootstrap with 9093, Do I need to disable the following configuration and then call the producer.sh by using only the Cluster CA crt, Is this called as 1-way TLS authentication?

tls: authentication: type: tls

./bin/kafka-console-producer.sh --broker-list mm-backup-cluster-kafka-bootstrap:9093 --topic mm-src-cluster.mm2-topic \ --producer-property security.protocol=SSL \ --producer-property ssl.truststore.type=PKCS12 \ --producer-property ssl.truststore.password=123456 \ --producer-property ssl.truststore.location=/tmp/certs/cluster.truststore.p12

Need your inputs and help here.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/strimzi/strimzi-kafka-operator/issues/3273, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLFOR5BPOJ347R74JP5ZS3RZQ62JANCNFSM4OOUNV7A .

[vperi1730]

That sounds good, What will be the behaviour if I replace my truststore with Keystore(certificate of a kafkauser) in the above producer script?

I believe this time it will not be an ANONYMOUS user, instead, it will do a 1-way TLS auth using the user crt which I have supplied(something like producer.keystore.p12), Is my understanding correct?

[scholzj]

One way TLS is what you have would be for me server authentication ... The client authenticates the server but it self remains unauthenticated. If you want to use the keystore and authenticate the user, that would be the configuration you had originally.

On Thu, Jul 2, 2020, 14:47 vperi1730 notifications@github.com wrote:

That sounds good, What will be the behaviour if I replace my truststore with Keystore(certificate of a kafkauser) in the above producer script?

I believe this time it will not be an ANONYMOUS user, instead, it will do a 1-way TLS auth using the user crt which I have supplied(something like producer.keystore.p12), Is my understanding correct?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/strimzi/strimzi-kafka-operator/issues/3273#issuecomment-652984958, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLFOR34LVGZ7IHTDZGSTV3RZR6VZANCNFSM4OOUNV7A .

[vperi1730]

OK, thanks for the clarification. Closing this thread.

[vperi1730]

I have another question here, after enabling the tls: {}, Now i am able to hit the request with ANONYMOUS user, However, one of the issues I see is, ANONYMOUS doesn't have access to a particular topic.

Question is, Do we need to manually add the ACLs for the ANONYMOUS user, or are they enable by default?

 ./bin/kafka-console-producer.sh --broker-list mm-backup-cluster-kafka-bootstrap:9093 --topic mm-src-cluster.mm2-topic \
> --producer-property security.protocol=SSL \
> --producer-property ssl.truststore.type=PKCS12 \
> --producer-property ssl.truststore.password=123456 \
> --producer-property ssl.truststore.location=/tmp/certs/cluster.truststore.p12
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
>ok
[2020-07-03 06:39:11,222] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {mm-src-cluster.mm2-topic=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2020-07-03 06:39:11,223] ERROR [Producer clientId=console-producer] Topic authorization failed for topics [mm-src-cluster.mm2-topic] (org.apache.kafka.clients.Metadata)
[2020-07-03 06:39:11,224] ERROR Error when sending message to topic mm-src-cluster.mm2-topic with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [mm-src-cluster.mm2-topic]

here is my ACL's list which doesn't show any ANONYMOUS list.

./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=127.0.0.1:2181 --list
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Current ACLs for resource `Topic:LITERAL:*`:
        User:CN=client-producer-user has Allow permission for operations: Read from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Write from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Create from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Write from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Describe from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Create from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Write from hosts: *
        User:CN=client-producer-user has Allow permission for operations: Describe from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Read from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Read from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Describe from hosts: *
        User:CN=client-consumer-user has Allow permission for operations: Create from hosts: *

Current ACLs for resource `Group:LITERAL:connect-cluster`:
        User:CN=client-consumer-user has Allow permission for operations: All from hosts: *
        User:CN=mm02-prod-user has Allow permission for operations: Describe from hosts: *

2) For disabling tls in external does the following configuration looks correct??

external:
        tls: false
        type: loadbalancer

[vperi1730]

It worked, Closing the ticket.

[alokhom]

hi. @vperi1730 @scholzj how did you fix the "doesn't have access to a particular topic."part ? I have a topic my-topic, i have a user my-user with ACLs to access my-topic, the authentication is OK and it uses truststore(ca.crt) and keystore( user.p12) , user.password but access to the topic is not authorized. ".TopicAuthorizationException: Not authorized to access topics: [my-topic]"

[scholzj]

Can you share the YAML for the KafkaUser resource with the ACLs?

[alokhom]

Kafka spec

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  creationTimestamp: '2020-08-25T14:38:29Z'
  generation: 11
  name: my-cluster
  namespace: kafka
  resourceVersion: '2334765'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkas/my-cluster
  uid: 984954e3-0c68-4ea0-a9e0-0d9e8f55f4a6
spec:
  entityOperator:
    topicOperator:
      reconciliationIntervalSeconds: 90
    userOperator:
      reconciliationIntervalSeconds: 120
  kafka:
    authorization:
      type: simple
    config:
      log.message.format.version: '2.5'
      offsets.topic.replication.factor: 3
      transaction.state.log.min.isr: 2
      transaction.state.log.replication.factor: 3
    listeners:
      external:
        authentication:
          type: tls
        overrides:
          bootstrap:
            host: bootstrap.apps.kafka.cluster42.openshift-enfo.se
          brokers:
            - broker: 0
              host: broker-0.apps.kafka.cluster42.openshift-enfo.se
            - broker: 1
              host: broker-1.apps.kafka.cluster42.openshift-enfo.se
            - broker: 2
              host: broker-2.apps.kafka.cluster42.openshift-enfo.se
        type: route
      plain:
        authentiation:
          type: scram-sha-512
      tls:
        authentiation:
          type: tls
    replicas: 3
    storage:
      class: rook-ceph-block
      size: 20Gi
      type: persistent-claim
    version: 2.5.0
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral

Kafka Topic spec -

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaTopic
metadata:
  creationTimestamp: '2020-08-25T14:41:13Z'
  generation: 1
  labels:
    strimzi.io/cluster: my-cluster
  name: my-topic
  namespace: kafka
  resourceVersion: '1468960'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkatopics/my-topic
  uid: 52d65111-ba4f-49ad-bb56-1db93b35d60b
spec:
  config:
    retention.ms: 604800000
    segment.bytes: 1073741824
  partitions: 10
  replicas: 3

kafkaUser spec-

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  creationTimestamp: '2020-08-26T20:03:09Z'
  generation: 3
  labels:
    strimzi.io/cluster: my-cluster
  name: my-user
  namespace: kafka
  resourceVersion: '2336125'
  selfLink: /apis/kafka.strimzi.io/v1beta1/namespaces/kafka/kafkausers/my-user
  uid: ecab2527-000a-47cb-bae8-e24f31bd2406
spec:
  authentication:
    type: tls
  authorization:
    acls:
      - host: '*'
        operation: Read
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Describe
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Read
        resource:
          name: my-group
          patternType: literal
          type: group
      - host: '*'
        operation: Write
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Create
        resource:
          name: my-topic
          patternType: literal
          type: topic
      - host: '*'
        operation: Describe
        resource:
          name: my-topic
          patternType: literal
          type: topic
    type: simple

I applied TLS on external and internal listeners I extracted the CA.cert to make a truststore. I extracted the user.p12 and user.password to make the keystore i used groupID my-group for Kafka Consumer to read data from topic my-user it works now to use an external Camel route application to connect to a Openshift Strimzi !!!

Request: It would be great if you provided a portal for the strimzi from the operator which: 1) could show us data in the topics 2) help configure/select the CA/User files from the namespace 3) check the ZK 4) provide a console that would help run the Kafka CLI commands like listing ACLs / etc. Other commands of Kafka. 5) give us realtime/basic/5 min old analytics 6) add/remove brokers make it a more easy for non Kafka technicians to do some day-2 operations. etc.