search

tchiotludo / akhq

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...
https://akhq.io/
Apache License 2.0
3.34k stars 646 forks source link

Issue with SSL_SASL and kerberos authentication SSL handshake failure #1236

Closed Duella12345 closed 1 year ago

Duella12345 commented 1 year ago

Hello ,

I am trying to connect to akhq with a kerberos backed Kafka cluster, we were previously connected just with SSL. Below are setup details

Application.yml Configuration:

micronaut:
  security:
    enabled: true
    # Ldap authentication configuration
    ldap:
      default:
        enabled: true
        context:
          server: 'ldap:/xxx:389'
          managerDn: 'cn=searchit,ou=functional,ou=btplc,dc=iuser,dc=iroot,dc=adidom,DC=com'
          managerPassword: {{ldap_manager_password}}
        search:
          base: "OU=btplc,DC=iuser,DC=iroot,DC=adidom,DC=com"
          filter: "cn={0}"
        groups:
          enabled: true
          #base: "OU=group,OU=rt_datameer,OU=application,OU=btplc,dc=iuser,dc=iroot,dc=adidom,dc=com"
          base: "CN=Users,DC=iuser,DC=iroot,DC=adidom,DC=com"
          filter: "member={0}"
  server:
    context-path: "/kafka-ui"
    port: 9090
    host: "0.0.0.0"

logger:
  levels:
    # Disable SSL handshake failed error logs
    org.apache.kafka.common.errors.SslAuthenticationException: ERROR
    org.apache.kafka.clients.admin.internals.AdminMetadataManager: ERROR

akhq:
  server:
    base-path: "/kafka-ui/"
    access-log: # Access log configuration (optional)
      enabled: true # true by default
      name: org.akhq.log.access # Logger name
      format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

  #Default kafka properties for each clients, available for admin / producer / consumer (optional)
  clients-defaults:
    consumer:
      properties:
        isolation.level: read_committed
        default.api.timeout.ms: 60000
  connections:
    kafkacluster:
      properties:
        bootstrap.servers: "{{bootstrap_servers}}"
        security.protocol: SASL_SSL
        ssl.truststore.type: "JKS"
        ssl.truststore.location: "/app/certs/kafka.server.truststore.jks"
        ssl.truststore.password: {{ssl_truststore_password}}
        sasl.mechanism: GSSAPI
        sasl.jaas.config: com.sun.security.auth.module.Krb5LoginModule required renewTGT=false doNotPrompt=true useKeyTab=true useTicketCache=false storeKey=true debug=true keyTab="/app/certs/client.keytab" serviceName="kafka" principal="{{principal}}";

  #Auth & Roles (optional)
  security:
    default-group: readonly # Default groups for all the user even unlogged user
    # Groups definition
    groups:
      - name: "readonly"
          #roles:
        #- topic/read
        attributes:
          topic-filter-regexp: "2387437.*"
      - name: "admin"
        roles:  # roles for the group
          - topic/read
          - topic/insert
          - topic/delete
          - topic/config/update
          - node/read
          - node/config/update
          - topic/data/read
          - topic/data/insert
          - topic/data/delete
          - group/read
          - group/delete
          - group/offsets/update
          - registry/read
          - registry/insert
          - registry/update
          - registry/delete
          - registry/version/delete
          - acls/read
          - connect/read
          - connect/insert
          - connect/update
          - connect/delete
          - connect/state/update
        attributes:
          # Regexp to filter topic available for group
          topics-filter-regexp: ".*"
      - name: "topic-reader"
        roles:
          - topic/read
          - topic/insert
          - topic/config/update
          - topic/data/read
          - topic/data/insert
          - topic/data/delete
        attributes:
          topics-filter-regexp: ".*"
    # Ldap Groups configuration (when using ldap)
    ldap:
      default-group: "readonly"
      groups:
        - name: "git_user_bucket"
          groups: # Akhq groups list
            - admin
        - name: "Domain Users"
          groups:
            - "topic-reader"
      users:
        - username: xxx
          groups: # Akhq groups list
            - admin
        - username:xxx
          groups:
            - admin
        - username: xxx
          groups:
            - admin
        - username: xxx
          groups:
            - admin

When We start akhq we are getting below error :

2022-10-19 15:25:02,657 INFO  main       i.m.runtime.Micronaut      Startup completed in 2516ms. Server Running: http://0.0.0.0:9090
[ansible@xxx ~]$ docker logs kafka-ui
2022-10-19 15:25:02,657 INFO  main       i.m.runtime.Micronaut      Startup completed in 2516ms. Server Running: http://0.0.0.0:9090
2022-10-19 15:32:15,344 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:15.246433Z] [Duration: 95 ms] [Url: GET /kafka-ui/] [Status: 307] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,497 INFO  pGroup-1-2 org.akhq.log.access        [Date: 2022-10-19T15:32:15.496799Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,692 INFO  pGroup-1-2 org.akhq.log.access        [Date: 2022-10-19T15:32:15.692491Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/css/main.391157bf.chunk.css] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,754 INFO  pGroup-1-3 org.akhq.log.access        [Date: 2022-10-19T15:32:15.754601Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/css/2.7caccc14.chunk.css] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,783 INFO  pGroup-1-4 org.akhq.log.access        [Date: 2022-10-19T15:32:15.783327Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/js/2.62ae1d40.chunk.js] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,796 INFO  pGroup-1-5 org.akhq.log.access        [Date: 2022-10-19T15:32:15.796414Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/js/main.2631d833.chunk.js] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,889 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:15.886171Z] [Duration: 3 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:15,965 INFO  pGroup-1-6 org.akhq.log.access        [Date: 2022-10-19T15:32:15.964845Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/media/icon.648ce9c8.svg] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:16,062 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:16.062199Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/auths] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:16,077 INFO  pGroup-1-7 org.akhq.log.access        [Date: 2022-10-19T15:32:16.077511Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/media/fontawesome-webfont.af7ae505.woff2] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:16,089 INFO  pGroup-1-9 org.akhq.log.access        [Date: 2022-10-19T15:32:16.089016Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/static/media/logo.45903e1f.svg] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:16,234 INFO  pGroup-1-8 org.akhq.log.access        [Date: 2022-10-19T15:32:16.234112Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui/manifest.json] [Status: 200] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:19,208 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:18.909875Z] [Duration: 298 ms] [Url: POST /kafka-ui/login] [Status: 303] [Ip: /xxx] [User: Anonymous]
2022-10-19 15:32:19,362 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:19.361516Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,407 INFO  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:19.405841Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/cluster] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,474 INFO  Group-1-10 org.akhq.log.access        [Date: 2022-10-19T15:32:19.474559Z] [Duration: 0 ms] [Url: GET /kafka-ui/ui] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,552 INFO  -thread-11 org.akhq.log.access        [Date: 2022-10-19T15:32:19.552626Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/cluster] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,572 INFO  -thread-11 org.akhq.log.access        [Date: 2022-10-19T15:32:19.571678Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,617 INFO  -thread-11 org.akhq.log.access        [Date: 2022-10-19T15:32:19.616739Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: 612241921]
Debug is  true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is null isInitiator true KeyTab is /app/certs/client.keytab refreshKrb5Config is false principal is kafka/xxx@xxx tryFirstPass is false useFirstPass is false storePass is false clearPass is false
2022-10-19 15:32:19,715 INFO  -thread-12 org.akhq.log.access        [Date: 2022-10-19T15:32:19.714947Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:19,779 INFO  -thread-12 org.akhq.log.access        [Date: 2022-10-19T15:32:19.778663Z] [Duration: 0 ms] [Url: GET /kafka-ui/api/me] [Status: 200] [Ip: /xxx] [User: 612241921]
principal is kafka/xxx@xxx
Will use keytab
Commit Succeeded

2022-10-19 15:32:20,031 WARN  -thread-11 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.location' was supplied but isn't a known config.
2022-10-19 15:32:20,031 WARN  -thread-11 .a.k.c.a.AdminClientConfig The configuration 'sasl.jaas.config' was supplied but isn't a known config.
2022-10-19 15:32:20,031 WARN  1-thread-5 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.location' was supplied but isn't a known config.
2022-10-19 15:32:20,032 WARN  -thread-11 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.password' was supplied but isn't a known config.
2022-10-19 15:32:20,032 WARN  1-thread-5 .a.k.c.a.AdminClientConfig The configuration 'sasl.jaas.config' was supplied but isn't a known config.
2022-10-19 15:32:20,032 WARN  -thread-11 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.type' was supplied but isn't a known config.
2022-10-19 15:32:20,032 WARN  1-thread-5 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.password' was supplied but isn't a known config.
2022-10-19 15:32:20,032 WARN  1-thread-5 .a.k.c.a.AdminClientConfig The configuration 'ssl.truststore.type' was supplied but isn't a known config.
2022-10-19 15:32:20,141 WARN  inclient-1 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-1] Connection to node -1 (xxx/xxx:9093) could not be established. Broker may not be available.
2022-10-19 15:32:20,602 WARN  inclient-2 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-2] Connection to node -2 (xxx.xxx/xxx:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.
2022-10-19 15:32:20,610 ERROR inclient-1 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-1] Connection to node -2 (xxx.xxx/xxx:9093) failed authentication due to: SSL handshake failed
2022-10-19 15:32:20,614 WARN  1-thread-5 org.akhq.log.access        [Date: 2022-10-19T15:32:19.487644Z] [Duration: 1126 ms] [Url: GET /kafka-ui/api/kafkacluster/topic] [Status: 500] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:20,629 WARN  inclient-1 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-1] Connection to node -1 (xxx/xxx:9093) could not be established. Broker may not be available.
2022-10-19 15:32:20,663 ERROR inclient-2 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-2] Connection to node -3 (xxx.xxx/10.13.148.250:9093) failed authentication due to: SSL handshake failed
2022-10-19 15:32:20,666 WARN  -thread-11 org.akhq.log.access        [Date: 2022-10-19T15:32:19.645943Z] [Duration: 1020 ms] [Url: GET /kafka-ui/api/kafkacluster/topic] [Status: 500] [Ip: /xxx] [User: 612241921]
2022-10-19 15:32:20,674 ERROR inclient-1 o.a.k.c.NetworkClient      [AdminClient clientId=adminclient-1] Connection to node -3 (xxx.xxx/10.13.148.250:9093) failed authentication due to: SSL handshake failed
tchiotludo commented 1 year ago

We don't support such old version of akhq, please update to latest version and reopen if you have the issues