Questions about kafka bridge scram-sha-512 authentication

[dilox]

Dear Strimzi, I have some doubts reguarding kafka bridge. I have an external kafka listener with this configuration:

...
    authorization:
      type: simple
      superUsers:
        - bridgeuser
...
      external:
        type: loadbalancer
        configuration:
          brokerCertChainAndKey:
            secretName: kafka-external-n
            certificate: n.pem
            key: n.pem
        tls: true
        authentication:
          type: scram-sha-512
          #type: tls
        overrides:
          brokers:
          - broker: 0
            loadBalancerIP: apimichele-kafka.n.io
            advertisedHost: IP

and it's working fine. I can produce and consume with kafka java class passing username e password into the jaas.conf (let's encrypt certificate is working as expected, tnx to strimzi 17.0 :+1: ).

Now they ask me an http bridge...

So I deployed kafka bridge and if it's deployed trough the internal/plain listener no problem.

But when I try to connect through the external ip, this is the log I receive:

2020-05-08 14:28:06,148 WARN [Producer clientId=producer-1] Connection to node -1 (xxxx:9094) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient) [kafka-producer-network-thread | producer-1]

I think the secret is fine.. this is the bridge configuration:

---
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaBridge
metadata:
  name: kafka-external
namespace: openwhisk
spec:
  replicas: 1
  bootstrapServers: apimichele-kafka..io:9094
  http:
    port: 8080
  authentication:
    type: scram-sha-512
    username: bridgeuser
    passwordSecret:
      secretName: bridgeuser
      password: password

I include also this:

$ ko get secret bridgeuser -o yaml
apiVersion: v1
data:
  password: xxxxXXXXXXxxxx
kind: Secret
metadata:
  creationTimestamp: "2020-05-08T14:10:46Z"
  labels:
    app.kubernetes.io/instance: bridgeuser
    app.kubernetes.io/managed-by: strimzi-user-operator
    app.kubernetes.io/name: strimzi
    strimzi.io/cluster: kafka-external
    strimzi.io/kind: KafkaUser
  name: bridgeuser
  namespace: openwhisk

I tried other configuration with no luck, can you help me, or point me to the right documenation, please? p.s: I am using strimzi 17.

[scholzj]

I think there are several issues which could cause problems here:

• Your Kafka cluster has the loadbalancer interface with TLS encryption. But you didn't configured that in the KafkaBridge. Please have a look here: https://strimzi.io/docs/latest/full.html#assembly-kafka-bridge-tls-deployment-configuration-kafka-bridge ... you would need to add something like this:

  tls:
    trustedCertificates:
    - secretName: kafka-external-n
      certificate: n.pem

• In the external listener configuration, you have the custom TLS certificate:

      configuration:
        brokerCertChainAndKey:
          secretName: kafka-external-n
          certificate: n.pem
          key: n.pem

  Normally when I use is I have always different files for the public key (certificate field) and private key (key option). You seem to use one file for both - I don't think we ever tried it ... it might work it might not.

• The overrides section in the Kafka broker looks weird:

      overrides:
        brokers:
        - broker: 0
          loadBalancerIP: apimichele-kafka.n.io
          advertisedHost: IP

  But again, without knowing your infrastructure, maybe that works but it looks weird to me.

[dilox]

Great scholzj!

  tls:
    trustedCertificates:
    - secretName: kafka-external-n
      certificate: n.pem

was exactly what I missed and everything is ok now. p.s: sure, I made a mistake coping and redacting the overriding part (perhaps I get from the kubectl description), this is how it looks:

        overrides:
          brokers:
          - broker: 0
            loadBalancerIP: apimichele-kafka.n.io
            advertisedHost: 35.xx.xxx.xx9

Thanks again, Pierluigi

[dilox]

Dear strimzi, how do I have to plain handling multiple kafka users on different kafka bridge? I mean.. setting a superuser as bridge users, everyone having the right url to curl as access on every topic. I thought to set up http authentication on the various proxy_pass section of the nginx.conf, but before starting experimentations I would like to know your opinion/suggestions, maybe you already have some example configuration and perhaps the api itself already handle authentication. Can you point me to some documentations about it? Thanks in advance, Pierluigi.

[scholzj]

The Bridge it self currently connects only as single user for everyone. It doesn't have to be a super users - you can use regular user and in that case it would be limited to do whatever the user can do.

To secure it on better granularity you have basically two options:

• Deploy a bridge per user with limited rights
• Use some security proxy infront of the bridge to do its own authentication and authorization. You can use proxies such as Nginx or API Gateways for example.

[dilox]

Thanks scholtz, yes, multiple bridges + multiple authenticated proxies (nginx) is my choice.

[ppatierno]

@dilox in that regard I did a little bit of "exploration" time ago using Nginx. There are simple configuration examples here: https://github.com/ppatierno/nginx-strimzi-kafka-bridge

[dilox]

thanks a lot @ppatierno ! the authentication example in your configurations is what I need