search

tchiotludo / akhq

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...
https://akhq.io/
Apache License 2.0
3.37k stars 653 forks source link

Hi trying to add ldap config for the below, as user or with our ID unable to login could you please help #479

Closed Ranganath9979 closed 3 years ago

tchiotludo commented 3 years ago

Debuging ldap connection can be done with

curl -i -X POST -H "Content-Type: application/json" \
       -d '{ "configuredLevel": "TRACE" }' \
       http://localhost:8080/loggers/io.micronaut.configuration.security

and also please format the yaml with triple `

Ranganath9979 commented 3 years ago

HI above is config yml file trying to set up ldap config, getting error can you please help

tchiotludo commented 3 years ago

missing jwt configuration : https://github.com/tchiotludo/akhq#jwt

or use the new dev image that provide a default one

Ranganath9979 commented 3 years ago

Hi thought to set basic auth with my ID still unable to is my config right, what Ami is missing here?

tchiotludo commented 3 years ago

still missing the jwt token like that : 

micronaut:
  security:
    enabled: true
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: <Your secret here>
Ranganath9979 commented 3 years ago

Ok I'm not sure about jwt do we need to generate tokens

tchiotludo commented 3 years ago

You need to create a secret (or use default one on dev image) in order to encode the session cookie (stored as jwt token). Without that, the login can't work

Ranganath9979 commented 3 years ago

I'm trying to set up ldap config Ui is coming up unable to login with ID could you please advise?

tchiotludo commented 3 years ago

Can you enable debug with that : 

curl -i -X POST -H "Content-Type: application/json" \
       -d '{ "configuredLevel": "TRACE" }' \
       http://localhost:8080/loggers/io.micronaut.configuration.security

Also remove secret from your post please 😅

Ranganath9979 commented 3 years ago

I deleted thanks, when I run the above I'm getting 200 Ok still unable to login

Ranganath9979 commented 3 years ago

HTTP/1.1 200 OK Date: Tue, 24 Nov 2020 06:27:38 GMT connection: keep-alive transfer-encoding: chunked

tchiotludo commented 3 years ago

It will only enable a verbose log for ldap. Now look at the log if akhq to have some insight

Ranganath9979 commented 3 years ago

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source) at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source) at java.base/java.security.cert.CertPathBuilder.build(Unknown Source) ... 186 common frames omitted

tchiotludo commented 3 years ago

You are using ldaps ?

If yes, you need to add the certificate like here: https://github.com/tchiotludo/akhq/issues/226#issuecomment-589811445

Ranganath9979 commented 3 years ago

java -Djavax.net.ssl.keyStore=MyKeystoreFile \ -Djavax.net.ssl.keyStorePassword=mysecret \ External

Hi I need to add above in script?

Ranganath9979 commented 3 years ago

I mean in application.yml file right?

tchiotludo commented 3 years ago

no in the script launch, depending if you are using docker or jar, but please provide more information if you want help ...

Ranganath9979 commented 3 years ago

I was trying to set up ldap config Ui is coming up unable to login with ID? and logs getting this error

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(Unknown Source) at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source) at java.base/java.security.cert.CertPathBuilder.build(Unknown Source) ... 186 common frames omitted

U referred me the above link where I need to add the certificate, could you help with this

Ranganath9979 commented 3 years ago

Currently I'm using docker but I need to update cookbook script later I think I need use jar file? which is later though thinking to finish with this?

tchiotludo commented 3 years ago

so if you are using docker, you need to use JAVA_OPTS env var with value -Djavax.net.ssl.keyStore=MyKeystoreFile -Djavax.net.ssl.keyStorePassword=mySecret and mount the sslKeystore in your docker image, changing the MyKeystoreFile and mySecret with your values

Ranganath9979 commented 3 years ago

Hi After adding JAva Opts -Djavax.net.ssl.keyStore=MyKeystoreFile -Djavax.net.ssl.keyStorePassword=mySecret

it is not starting what went wrong now, it was on after adding these options docker is not turning on could you please help

tchiotludo commented 3 years ago

You should replace the My.... Also please provide a true report, this is not working is not a report :sob:

Please provide :

Ranganath9979 commented 3 years ago

Hi

2020-12-08 15:59:53,239 DEBUG pGroup-1-5 LdapAuthenticationProvider Starting authentication with configuration [default] 2020-12-08 15:59:53,239 DEBUG pGroup-1-5 LdapAuthenticationProvider Attempting to initialize manager context 2020-12-08 15:59:53,243 DEBUG pGroup-1-5 LdapAuthenticationProvider Manager context initialized successfully 2020-12-08 15:59:53,243 DEBUG pGroup-1-5 LdapAuthenticationProvider Attempting to authenticate with user [a1509893] 2020-12-08 15:59:53,248 DEBUG pGroup-1-5 LdapAuthenticationProvider User found in context [CN=Chandrashekarappa\, Ranganatha 1509893,OU=Default,OU=Users,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com]. Attempting to bind. 2020-12-08 15:59:53,604 DEBUG pGroup-1-5 LdapAuthenticationProvider Successfully bound user [CN=Chandrashekarappa\, Ranganatha 1509893,OU=Default,OU=Users,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com]. Attempting to retrieving groups. 2020-12-08 15:59:53,970 DEBUG pGroup-1-5 .DefaultLdapGroupProcessor No groups found! 2020-12-08 15:59:53,971 DEBUG pGroup-1-5 .DefaultLdapGroupProcessor Attempting to add any additional groups... 2020-12-08 15:59:53,971 DEBUG pGroup-1-5 LdapAuthenticationProvider Group search returned [[]] for user [a1509893] 2020-12-08 15:59:53,971 TRACE pGroup-1-5 LdapAuthenticationProvider Attempting to map [a1509893] with groups [[]] to an authentication response. 2020-12-08 15:59:53,971 DEBUG pGroup-1-5 LdapAuthenticationProvider Response successfully created for [a1509893]. Response is authenticated: [true] 2020-12-08 15:59:53,973 INFO pGroup-1-5 org.akhq.log.access [Date: 2020-12-08T15:59:53.23656Z] [Duration: 736 ms] [Url: POST /login] [Status: 303] [Ip: 0] [User: 172.23.32.202]

Ranganath9979 commented 3 years ago

I'm getting this error what changes I have to make?

Ranganath9979 commented 3 years ago

you wanna C my .yml file?

tchiotludo commented 3 years ago

It's seems to be your groups.base that don't match any group for your users. I don't have a lot of experience with LDAP and all the configuration that you can provide is here : https://micronaut-projects.github.io/micronaut-security/latest/guide/#ldap

Maybe you could find another people that have successfully use your ldap since it's always the same config for all tools

Ranganath9979 commented 3 years ago

Hi, My issue with Ldap is resolved...now without login also others in the group can be able to open Ui and can make changes and even delete topics what changes I have to made....

Ui should respond only through login what I have to do?

tchiotludo commented 3 years ago

:tada: https://github.com/tchiotludo/akhq#security : 

akhq:
  security:
    default-group: no-roles
Ranganath9979 commented 3 years ago

it worked thanks for now....

we are setting with, test cluster may need to set ssl there is it ok check, if reqd. anything?

tchiotludo commented 3 years ago

sorry didn't understand

Ranganath9979 commented 3 years ago

Hi I'm trying disable the option of creating/deleting topics from Akhq Ui could some please help with it?

Ranganath9979 commented 3 years ago

I fixed it thankyou

Ranganath9979 commented 3 years ago

I was trying to set standalone, what is the startup script that can be used to start application.yml when we are using ldap config. is it same java -Dmicronaut.config.files=/path/to/application.yml -jar akhq.jar

Ranganath9979 commented 3 years ago

sudo docker run -d --env "JAVA_OPTS=-Djavax.net.ssl.trustStore=/app/truststore.jks -Djavax.net.ssl.trustStorePassword=xxxxxxx" -p 8080:8080 -v /tmp/application.yml:/app/application.yml -v /tmp/trust.jks:/app/truststore.jks 456de460c95

This I'm using to start from docker, I need help with standalone?

tchiotludo commented 3 years ago

java -Djavax.net.ssl.trustStore=/app/truststore.jks -Djavax.net.ssl.trustStorePassword=xxxxxxx -Dmicronaut.config.files=/path/to/application.yml -jar akhq.jar must do the trick

Ranganath9979 commented 3 years ago

🎉 https://github.com/tchiotludo/akhq#security :

akhq:
  security:
    default-group: no-roles

Hi my issue was fixed I was bale to login earlier, all of sudden I'm not able to login what shall I check, and below is my yml file

akhq:
  security:
    default-group: no-roles # Default groups for all the user even unlogged user
    # Groups definition
    groups:
      admin: # Group name
        roles:  # roles for the group
          - topic/read
          - node/read
          - topic/data/read
          - group/read
          - group/offsets/update
          - registry/read
          - registry/version/delete
tchiotludo commented 3 years ago

@Ranganath9979: use the dev image,, we have changed the format of the configuration in last release and it's was a wrong idea. So we revert on dev one

Ranganath9979 commented 3 years ago

will try thanks

Ranganath9979 commented 3 years ago 
micronaut:
  security:
    enabled: true
    # Ldap authentificaton configuration
    ldap:
      default:
        enabled: true
        context:
          server: 'lXXXXXXXXXXXXXXXXXXXXXX'
          managerDn: 'cn=BBY-S-KAFKAHQ-DEV,OU=System,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com'
          managerPassword: 'XXXXXXXXXXXXXXX'
        search:
          enabled: true
          base: "OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com"
          filter: "sAMAccountName={0}"
        groups:
          enabled: true
          subtree: true
          base: "OU=Groups,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com"
          filter: "member={0}"
akhq:
  security:
    default-group: no-roles # Default groups for all the user even unlogged user
    # Groups definition
    groups:
      admin: # Group name
        roles:  # roles for the group
          - topic/read
          - node/read
          - topic/data/
          - group/read
          - group/offsets/update
          - registry/read
          - registry/version/

    ldap:
      group:
        BBY-U-ENT-KAFKA:
          groups:
            - admin
  server:
    access-log: # Access log configuration (optional)
      enabled: true # true by default
      name: org.akhq.log.access # Logger name
      format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

  # default kafka properties for each clients, available for admin / producer / consumer (optional)
  clients-defaults:
    consumer:
      properties:
        isolation.level: read_committed

  # list of kafka cluster available for akhq
  connections:
    my-cluster-plain-text: # url friendly name for the cluster (letter, number, _, -, ... dot are not allowed here)
      properties: # standard kafka properties (optional)
        bootstrap.servers: "XXXXXXXXXXXXX:9092"
      schema-registry:
        url: "XXXXXXXXXXXX:8081"
  pagination:
    page-size: 25 # number of elements per page (default : 25)
    threads: 16 # Number of parallel threads to resolve page

  # Topic list display options (optional)
  topic:
    retention: 172800000 # default retention period when creating topic
    partition: 3 #  default number of partition when creating topic
    replication: 3 # default number of replicas when creating topic
    default-view: ALL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM)
    internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)
      - "^_.*$"
      - "^.*_schemas$"
      - "^.*connect-config$"
      - "^.*connect-offsets$1"
      - "^.*connect-status$"
    stream-regexps: # list of regexp to be considered as internal stream topic
      - "^.*-changelog$"
      - "^.*-repartition$"
      - "^.*-rekey$"
    skip-consumer-groups: false # Skip loading consumer group information when showing topics

  # Topic display data options (optional)
  topic-data:
    sort:  # default sort order (OLDEST, NEWEST) (default: OLDEST)
    size: 50 # max record per page (default: 50)
    poll-timeout: 1000 # The time, in milliseconds, spent waiting in poll if data is not available in the buffer.

  # Auth & Roles (optional)
  # Ldap Groups configuration (when using ldap)
Ranganath9979 commented 3 years ago

Hey, this is my old application.yml I was using it was working fine, now unable to login as per your instructions I used latest dev image still no luck ,could you please help me right identations

tchiotludo commented 3 years ago

I've release a new version yesterday, no need for dev image now

Ranganath9979 commented 3 years ago

mean directly I can use application.yml file and configure it?

tchiotludo commented 3 years ago

Just make a try with latest version to be sure on the version that you are using

Ranganath9979 commented 3 years ago

still unable to login

Ranganath9979 commented 3 years ago

getting below error, for the above .yml file

java.lang.NullPointerException at io.micronaut.web.router.resource.ReactStaticResourceResolver.resolve(ReactStaticResourceResolver.java:34) at io.micronaut.http.server.netty.RoutingInBoundHandler.matchFile(RoutingInBoundHandler.java:630) at io.micronaut.http.server.netty.RoutingInBoundHandler.channelRead0(RoutingInBoundHandler.java:557) at io.micronaut.http.server.netty.RoutingInBoundHandler.channelRead0(RoutingInBoundHandler.java:140) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.micronaut.http.netty.stream.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:196) at io.micronaut.http.netty.stream.HttpStreamsServerHandler.channelRead(HttpStreamsServerHandler.java:121) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)

Ranganath9979 commented 3 years ago

Hi 2021-02-18 21:29:09,363 ERROR pGroup-1-2 .s.n.RoutingInBoundHandler Unexpected error occurred: Couldn't find any clusters on your configuration file, please ensure that the configuration file is loaded correctly

I'm getting below error when I'm starting, I tried with 0.17 version aswell no luck could you please help, I'm posting my yml file aswell

Ranganath9979 commented 3 years ago 
micronaut:
  security:
    enabled: true
    # Ldap authentificaton configuration
    ldap:
      default:
        enabled: true
        context:
          server: 'ldaps://adldap.na.bestbuy.com:636'
          managerDn: 'cn=BBY-S-KAFKAHQ-DEV,OU=System,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com'
          managerPassword: ''
        search:
          enabled: true
          base: "OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com"
          filter: "sAMAccountName={0}"
        groups:
          enabled: true
          subtree: true
          base: "OU=Groups,OU=Best Buy,OU=Users and Groups,DC=na,DC=bestbuy,DC=com"
          filter: "member={0}"

  server:
    context-path: "/akhq" # if behind a reverse proxy, path to akhq without trailing slash (optional). Example: akhq is
                     # behind a reverse proxy with url http://my-server/akhq, set base-path: "/akhq".
                     # Not needed if you're behind a reverse proxy with subdomain http://akhq.my-server/

    access-log: # Access log configuration (optional)
      enabled: true # true by default
      name: org.akhq.log.access # Logger name
      format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

  # default kafka properties for each clients, available for admin / producer / consumer (optional)
  clients-defaults:
    consumer:
      properties:
        isolation.level: read_committed

   # list of kafka cluster available for akhq
  connections:
    my-cluster-plain-text: # url friendly name for the cluster (letter, number, _, -, ... dot are not allowed here)
      properties: # standard kafka properties (optional)
        bootstrap.servers: "dtl01s8jmq01c.na.bestbuy.com:9092"
      schema-registry:
        url: "http://dtl01s8jmq01c.na.bestbuy.com:8081"
  pagination:
    page-size: 25 # number of elements per page (default : 25)
    threads: 16 # Number of parallel threads to resolve page

  # Topic list display options (optional)
  topic:
    retention: 172800000 # default retention period when creating topic
    partition: 3 #  default number of partition when creating topic
    replication: 3 # default number of replicas when creating topic
    default-view: ALL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM)
    internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)
      - "^_.*$"
      - "^.*_schemas$"
      - "^.*connect-config$"
      - "^.*connect-offsets$1"
      - "^.*connect-status$"
    stream-regexps: # list of regexp to be considered as internal stream topic
      - "^.*-changelog$"
      - "^.*-repartition$"
      - "^.*-rekey$"
    skip-consumer-groups: false # Skip loading consumer group information when showing topics

  # Topic display data options (optional)
  topic-data:
    sort:  # default sort order (OLDEST, NEWEST) (default: OLDEST)
    size: 50 # max record per page (default: 50)
    poll-timeout: 1000 # The time, in milliseconds, spent waiting in poll if data is not available in the buffer.

  # Auth & Roles (optional)
  akhq:
  security:
    default-group: no-roles # Default groups for all the user even unlogged user
    # Groups definition
    groups:
      admin: # Group name
        roles:  # roles for the group
          - topic/read
          - node/read
          - topic/data/read
          - group/read
          - group/offsets/update
          - registry/read
          - registry/version/delete

    ldap:
      group:
        BBY-U-ENT-KAFKA:
          groups:
            - admin