Kafka-UI is not showing the cluster info

[rsurenderreddy]

Issue submitter TODO list

• [X] I've looked up my issue in FAQ
• [X] I've searched for an already existing issues here
• [X] I've tried running main-labeled docker image and the issue still persists there
• [X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

I tried deploying kafka-UIon podman where I have provisioned the image. I have completed all the set up steps and while accessing the web page of kafka-UI it is not showing my cluster details.

i am able to get cluster details with MSK 2.8.1 on kafka ui but new msk version 3.5.1 is having issue.

i am using AWS MSK version of kafka 3.5.1 with IAM SASL method.

Expected behavior

it supposed to get cluster details on UI page

i checked /api/clsusters , it get empty reposnse back to UI

Your installation details

APP: v1.0.0

Steps to reproduce

test with msk 3.5.1 versiom

Screenshots

No response

Logs

No response

Additional context

No response

[github-actions[bot]]

Hi rsurenderreddy! 👋

Welcome, and thank you for opening your first issue in the repo!

Please wait for triaging by our maintainers.

As development is carried out in our spare time, you can support us by sponsoring our activities or even funding the development of specific issues. Sponsorship link

If you plan to raise a PR for this issue, please take a look at our contributing guide.

[Haarolean]

Please share the logs and the config

[kapybro[bot]]

Further user feedback is requested. Please reply within 7 days or we might close the issue.

[rsurenderreddy]

attached logs and config file

(https://gi[kafka.zip](https://github.com/user-attachments/files/16023904/kafka.zip)

[kapybro[bot]]

Thanks for the additional feedback! We'll get back to your issue soon.

[Haarolean]

@rsurenderreddy

$ unzip kafka.zip
Archive:  kafka.zip
warning [kafka.zip]:  zipfile is empty

Please use https://gist.github.com

[kapybro[bot]]

Further user feedback is requested. Please reply within 7 days or we might close the issue.

[rsurenderreddy]

sorry my bad.. attached zip file

kafka.zip

[kapybro[bot]]

Thanks for the additional feedback! We'll get back to your issue soon.

[Haarolean]

Please provide the outputs of: hostname/api/authorization hostname/api/clusters

[rsurenderreddy]

Here is the output of api's

// https://kafkaui.xyz.com/api/clusters

[

]

// https://kafkaui.xyz.com/api/authorization

{ "rbacEnabled": true, "userInfo": { "username": "sss@xyz.com", "permissions": [ { "clusters": [ "kafka-cluster" ], "resource": "APPLICATIONCONFIG", "value": null, "actions": [ "VIEW", "EDIT" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CLUSTERCONFIG", "value": null, "actions": [ "VIEW", "EDIT" ] }, { "clusters": [ "kafka-cluster" ], "resource": "TOPIC", "value": ".", "actions": [ "VIEW", "CREATE", "EDIT", "DELETE", "MESSAGES_READ", "MESSAGES_PRODUCE", "MESSAGES_DELETE" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CONSUMER", "value": ".", "actions": [ "VIEW", "DELETE", "RESET_OFFSETS" ] }, { "clusters": [ "kafka-cluster" ], "resource": "SCHEMA", "value": ".", "actions": [ "VIEW", "CREATE", "DELETE", "EDIT", "MODIFY_GLOBAL_COMPATIBILITY" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CONNECT", "value": ".", "actions": [ "VIEW", "EDIT", "CREATE", "RESTART", "DELETE" ] }, { "clusters": [ "kafka-cluster" ], "resource": "KSQL", "value": null, "actions": [ "EXECUTE" ] }, { "clusters": [ "kafka-cluster" ], "resource": "ACL", "value": ".", "actions": [ "VIEW" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CLUSTERCONFIG", "value": null, "actions": [ "VIEW" ] }, { "clusters": [ "kafka-cluster" ], "resource": "TOPIC", "value": ".", "actions": [ "VIEW", "MESSAGES_READ" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CONSUMER", "value": ".", "actions": [ "VIEW" ] }, { "clusters": [ "kafka-cluster" ], "resource": "SCHEMA", "value": ".", "actions": [ "VIEW" ] }, { "clusters": [ "kafka-cluster" ], "resource": "CONNECT", "value": ".*", "actions": [ "VIEW" ] } ] } }

[hongbo-miao]

I am using MSK 3.7.x KRaft mode. Also, I am also having some issues seeing the cluster in the Kafbat UI.

I can confirm same IAM role permission (granted all readonly permissions for now) works well for AKHQ and Redpanda Console.

Here is my config

yamlApplicationConfig:
  kafka:
    clusters:
      - name: yaml
        bootstrapServers: b-1.xxx.xx.c11.kafka.us-west-2.amazonaws.com:9098,b-2.xxx.xx.c11.kafka.us-west-2.amazonaws.com:9098,b-3.xxx.xx.c11.kafka.us-west-2.amazonaws.com:9098
        properties:
          security.protocol: SASL_SSL
          sasl.mechanism: AWS_MSK_IAM
          sasl.jaas.config: software.amazon.msk.auth.iam.IAMLoginModule required;
          sasl.client.callback.handler.class: software.amazon.msk.auth.iam.IAMClientCallbackHandler
  auth:
    type: disabled
  management:
    health:
      ldap:
        enabled: false
serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxx4xxx:role/KafbatUIRole-hm-kafbat-ui

https://hm-kafbat-ui.example.com/api/authorization returns

{
"rbacEnabled": false,
"userInfo": null
}

https://hm-kafbat-ui.example.com/api/clusters returns

[
  {
    "name": "yaml",
    "defaultCluster": null,
    "status": "offline",
    "lastError": {
      "message": "Error while creating AdminClient for Cluster yaml",
      "stackTrace": "java.lang.IllegalStateException: Error while creating AdminClient for Cluster yaml\n\tat io.kafbat.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56)\n\tat reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3783)\n\tat reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)\n\tat reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258)\n\tat reactor.core.publisher.FluxMap$MapConditionalSubscriber.onError(FluxMap.java:265)\n\tat reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1886)\n\tat reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:340)\n\tat reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onError(MonoCacheTime.java:363)\n\tat reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:180)\n\tat reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:278)\n\tat reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187)\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)\n\tat reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base/java.lang.Thread.run(Thread.java:840)\nCaused by: org.apache.kafka.common.errors.SaslAuthenticationException: [4d37b931-0d78-44a1-bf2c-8ebbf55da91a]: Access denied\n"
    },
    "brokerCount": 0,
    "onlinePartitionCount": 0,
    "topicCount": 0,
    "bytesInPerSec": null,
    "bytesOutPerSec": null,
    "readOnly": false,
    "version": "Unknown",
    "features": []
  }
]

[rsurenderreddy]

i tested 3.5.1 with AKHQ UI, this is loading cluster

[hongbo-miao]

Hi @rsurenderreddy do you mean AKHQ didn't work with MSK 3.5.1 when you open this ticket, and it works now? Could you post the solution? Thanks

[rsurenderreddy]

I opened this ticket for kafbat UI not for Akhq ui project.

I was testing AkHQ UI as a alternative solution for time being.

[hongbo-miao]

I see, sorry, my bad. Didn't read closely. Yeah both AKHQ and Redpanda Console works for us as well for MSK 3.7.x KRaft mode. Hopefully Kafbat UI can support as well.

[Haarolean]

@hongbo-miao please raise a new issue, your case is a different issue

[hongbo-miao]

Thanks @Haarolean , I posted at https://github.com/kafbat/kafka-ui/discussions/483

[Haarolean]

@rsurenderreddy /api/clusters returning no clusters makes no sense tbh, considering it's present in the logs. Can you try disabling RBAC for testing purposes and checking again?

[rsurenderreddy]

issue with my rbac , it doesn't have cluster name tied with the role.

issue fixed now and thanks @Haarolean for the help