Github oauth2 and rbac do not work.

finda-yeongjo commented 7 months ago

Issue submitter TODO list

[X] I've looked up my issue in FAQ
[X] I've searched for an already existing issues here
[X] I've tried running master-labeled docker image and the issue still persists there
[X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

I tried to apply self-hosted GithubEnterprise server oauth while running kafka-ui container, but only "Invalid Credentials" is output and no other logs exist.

I will attach the symptoms and settings below.

The page I referenced

Expected behavior

Github oauth Successful

Your installation details

docker-compose.yml

version: '2'
services:
kafka-ui:
image: provectuslabs/kafka-ui:master
container_name: kafka-ui
ports:
  - "80:8080"
restart: always
environment:
  LOGGING_LEVEL_ROOT: 'error'
  DYNAMIC_CONFIG_ENABLED: 'true'
  SPRING_CONFIG_ADDITIONAL-LOCATION: /config.yaml
  # AUTH_TYPE: "LOGIN_FORM"
  # SPRING_SECURITY_USER_NAME: test
  # SPRING_SECURITY_USER_PASSWORD: testtest
  KAFKA_CLUSTERS_0_NAME: test
  KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: SOME_KAFKA_ENDPOINT:9092
  KAFKA_CLUSTERS_0_ZOOKEEPER: SOME_ZOOKEEPER_ENDPOINT:2181
  KAFKA_CLUSTERS_0_KAFKACONNECT_0_NAME: SOME_KAFKA_CONNECT_NAME
  KAFKA_CLUSTERS_0_KAFKACONNECT_0_ADDRESS: SOME_KAFKA_CONNECT_ENDPOINT
  KAFKA_CLUSTERS_1_NAME: test1
  KAFKA_CLUSTERS_1_BOOTSTRAPSERVERS: SOME_KAFKA_ENDPOINT:9092
  KAFKA_CLUSTERS_1_ZOOKEEPER: SOME_ZOOKEEPER_ENDPOINT:2181
  KAFKA_CLUSTERS_2_NAME: test2
  KAFKA_CLUSTERS_2_BOOTSTRAPSERVERS: SOME_KAFKA_ENDPOINT:9092
  KAFKA_CLUSTERS_2_ZOOKEEPER: SOME_ZOOKEEPER_ENDPOINT:2181
  KAFKA_CLUSTERS_3_NAME: test3
  KAFKA_CLUSTERS_3_BOOTSTRAPSERVERS: SOME_KAFKA_ENDPOINT:9092
  KAFKA_CLUSTERS_3_ZOOKEEPER: SOME_ZOOKEEPER_ENDPOINT:2181
volumes:
  - ./config.yaml:/config.yaml
  - ./dynamic_config.yaml:/etc/kafkaui/dynamic_config.yaml
  - ./upload:/etc/kafkaui/uploads

config.yaml

auth:
type: OAUTH2
oauth2:
client:
  github:
    provider: github
    clientId: MY_GITHUB_OAUTH_APP_CLIENT_ID
    clientSecret: MY_GITHUB_OAUTH_APP_CLIENT_SECRET
    scope: read:org
    user-name-attribute: login
    authorization-uri: https://OUR_GITHUB_SERVER_URL/login/oauth/authorize
    token-uri: https://OUR_GITHUB_SERVER_URL/login/oauth/access_token
    user-info-uri: https://OUR_GITHUB_SERVER_URL/api/v3/user
    custom-params:
      type: github
rbac:
roles:
- name: "admins"
  clusters:
    - test
    - test1
  subjects:
    - provider: oauth_github
      type: organization
      value: "OUR_ORG_NAME"
  permissions:
    - resource: applicationconfig
      actions: all

    - resource: clusterconfig
      actions: all

    - resource: topic
      value: ".*"
      actions: all

    - resource: consumer
      value: ".*"
      actions: all

    - resource: schema
      value: ".*"
      actions: all

    - resource: connect
      value: ".*"
      actions: all

    - resource: ksql
      actions: all

    - resource: acl
      actions: [ view ]

Steps to reproduce

docker-compose up -d OR
docker-compose restart

Screenshots

Logs

docker-compose logs

kafka-ui    | Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
kafka-ui    |  _   _ ___    __             _                _          _  __      __ _
kafka-ui    | | | | |_ _|  / _|___ _ _    /_\  _ __ __ _ __| |_  ___  | |/ /__ _ / _| |_____
kafka-ui    | | |_| || |  |  _/ _ | '_|  / _ \| '_ / _` / _| ' \/ -_) | ' </ _` |  _| / / _`|
kafka-ui    |  \___/|___| |_| \___|_|   /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
kafka-ui    |                                  |_|
kafka-ui    |
kafka-ui    | 2024-03-04 06:28:41,288 INFO  [main] c.p.k.u.u.DynamicConfigOperations: Dynamic config loaded from /etc/kafkaui/dynamic_config.yaml
kafka-ui    | 2024-03-04 06:28:41,312 INFO  [main] c.p.k.u.KafkaUiApplication: Starting KafkaUiApplication using Java 17.0.6 with PID 1 (/kafka-ui-api.jar started by kafkaui in /)
kafka-ui    | 2024-03-04 06:28:41,312 DEBUG [main] c.p.k.u.KafkaUiApplication: Running with Spring Boot v3.1.1, Spring v6.0.10
kafka-ui    | 2024-03-04 06:28:41,313 INFO  [main] c.p.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
kafka-ui    | 2024-03-04 06:28:47,282 DEBUG [main] c.p.k.u.s.SerdesInitializer: Configuring serdes for cluster test
kafka-ui    | 2024-03-04 06:28:47,296 DEBUG [main] c.p.k.u.s.SerdesInitializer: Configuring serdes for cluster test1
kafka-ui    | 2024-03-04 06:28:47,299 DEBUG [main] c.p.k.u.s.SerdesInitializer: Configuring serdes for cluster test2
kafka-ui    | 2024-03-04 06:28:47,301 DEBUG [main] c.p.k.u.s.SerdesInitializer: Configuring serdes for cluster test3
kafka-ui    | 2024-03-04 06:28:48,583 INFO  [main] c.p.k.u.c.a.BasicAuthSecurityConfig: Configuring LOGIN_FORM authentication.
kafka-ui    | 2024-03-04 06:28:49,840 INFO  [main] c.p.k.u.KafkaUiApplication: Started KafkaUiApplication in 10.115 seconds (process running for 11.412)
kafka-ui    | 2024-03-04 06:28:50,872 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test
kafka-ui    | 2024-03-04 06:28:50,873 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test1
kafka-ui    | 2024-03-04 06:28:51,040 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test2
kafka-ui    | 2024-03-04 06:28:51,040 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test3
kafka-ui    | 2024-03-04 06:28:51,914 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: test2
kafka-ui    | 2024-03-04 06:28:51,916 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: test3
kafka-ui    | 2024-03-04 06:28:52,167 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: test
kafka-ui    | 2024-03-04 06:28:53,105 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: test1
kafka-ui    | 2024-03-04 06:29:19,837 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test
kafka-ui    | 2024-03-04 06:29:19,836 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test2
kafka-ui    | 2024-03-04 06:29:19,837 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test3
kafka-ui    | 2024-03-04 06:29:19,838 DEBUG [parallel-2] c.p.k.u.s.ClustersStatisticsScheduler: Start getting metrics for kafkaCluster: test4

Additional context

No response

github-actions[bot] commented 7 months ago

Hello there finda-yeongjo! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

qgervacio commented 6 months ago

same here. in my case i can still login but result in UI is the registered clusters are gone. using kafka-ui:v0.7.1, chart version 0.7.5

...
- name: SPRING_CONFIG_ADDITIONAL-LOCATION
  value: /roles.yml
...

---

apiVersion: v1
data:
  roles.yml: |
    rbac:
      roles:
        - name: readonly
          clusters:
            - cluster-abc
          subjects:
            - provider: oauth_google
              type: domain
              value: example.com
          permissions:
            - resource: APPLICATIONCONFIG
              actions: [ VIEW ]
            - resource: CLUSTERCONFIG
              actions: [ VIEW ]
            - resource: TOPIC
              value: .*
              actions: 
                - VIEW
                - MESSAGES_READ
            - resource: CONSUMER
              value: .*
              actions: [ VIEW ]
            - resource: SCHEMA
              value: .*
              actions: [ VIEW ]
            - resource: CONNECT
              value: .*
              actions: [ VIEW ]
            - resource: ACL
              value: .* # throws error if removed. doc mentioned that this is not needed(?)
              actions: [ VIEW ]
kind: ConfigMap
metadata:
  name: roles

Haarolean commented 6 months ago

Hey folks, happy to help with your issues here: https://github.com/kafbat/kafka-ui Feel free to raise one there

provectus / kafka-ui