topics.regex configuration error

Hello,

I am adding topics.regex property to try to filter my topics connections: Feature-1: schema-registry: url: https://feature-1.sample.com type: confluent basic-auth-username: some_user basic-auth-password: some_password ui-options: topic-data: sort: NEWEST properties: bootstrap.servers: server-url.confluent.cloud:9092 security.protocol: SASL_SSL topics.regex: "^.feature-1.$" sasl.mechanism: PLAIN sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='some_SA_user' password='some_SA_password';

*** I have replaced sensitive data.

I am getting this error here:

May I get any comments from you? I am adding the property topics.regex in the wrong place? Is there any other option to do this topics filter approach?

Hello @josue-sanchez-wabtec. Please refer to the doc about authorization here to understand how permissions and resources filtering works. Check also the application.example.yml file to see how to setup a right configuration for clusters and authentication

@AlexisSouquiere I am checking but it seems to be related to groups. I am using just admin group. Is there any other way to filter topics by an attribute configuration inside the connections section?

No there is nothing like this in the connections section. You have to customize the admin group and use the patterns attribute to allow only the topics you want


    groups:
      admin:
        - role: node-admin
        - role: topic-admin
          patterns: ["^.feature-1.$"]
        - role: connect-admin
        - role: registry-admin
        - role: group-read
        - role: connect-cluster-read
        - role: ksqldb-admin

I have this configuration set but it looks like something could be wrong, I suspect it is the groups: section as if I remove it I can succesfullly log in to the akhq UI but now I am not able to do it. I used this past issue https://github.com/tchiotludo/akhq/issues/500 but it looks like this was already implemented since then. So it should be ok to overwrite the admin group.

Also I am not getting any logs with errors or even warnings after enabling the debug using:

curl -i -X POST -H "Content-Type: application/json" \ -d '{ "configuredLevel": "TRACE" }' \ http://localhost:28081/loggers/io.micronaut.security

curl -i -X POST -H "Content-Type: application/json" \ -d '{ "configuredLevel": "TRACE" }' \ http://localhost:28081/loggers/org.akhq.configs

UI looks corrupted too, not centered:

Logs logs.log

micronaut:
  security:
    enabled: true
akhq.security:
  default-group: no-roles
  basic-auth:
    - username: <sample-username>
      password: <sample-pasword>
      groups:
      - admin
  groups:
    admin:
      - role: node-admin
      - role: topic-admin
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: connect-admin
      - role: registry-admin
      - role: group-read
      - role: connect-cluster-read
      - role: ksqldb-admin

akhq:
  connections:
    feature-internal-all:
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <bootstrap-server-url>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='some-user-sa' password='some-password-sa';

    feature-1:
      schema-registry:
        url: <feature-1-schema-registry-url>
        type: confluent
        basic-auth-username: <schema-registry-user>
        basic-auth-password: <schema-registry-password>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <bootstrap-server-url>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='some-user-sa' password='some-password-sa';

    feature-2:
      schema-registry:
        url: <feature-2-schema-registry-url>
        type: confluent
        basic-auth-username: <schema-registry-user>
        basic-auth-password: <schema-registry-password>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <bootstrap-server-url>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='some-user-sa' password='some-password-sa';

    feature-3:
      schema-registry:
        url: <feature-3-schema-registry-url>
        type: confluent
        basic-auth-username: <schema-registry-user>
        basic-auth-password: <schema-registry-password>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <bootstrap-server-url>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='some-user-sa' password='some-password-sa';

Thank you so much for any suggestion you could give me @AlexisSouquiere

@AlexisSouquiere I also tried adding the patterns and clusters filters on each role but I had same behavior, not able to log in and no error on logs.

  groups:
    admin:
      - role: node-admin
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: topic-admin
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: connect-admin
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: registry-admin
      - role: group-read
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: connect-cluster-read
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]
      - role: ksqldb-admin
        patterns: [ "*-internal-2-*" ]
        clusters: [ "feature-*" ]

Are you using the 0.24.0 version ? I think that I'm seeing in the logs things that changes since then on the authentication side connectsFilterRegexp, topicFilterRegexp, etc.).

Doc is currently usable for the dev branch but if you want to deploy 0.24.0, please check doc on this revision https://github.com/tchiotludo/akhq/tree/c280b093116ca4d89e1480358fff6f48fd82d2ae/docs/docs/configuration and application.example.yml too https://github.com/tchiotludo/akhq/blob/c280b093116ca4d89e1480358fff6f48fd82d2ae/application.example.yml

Permissions management has been rewrited to have fine-grained permissions and especially cluster filtering based on permissions. If you need this, you should use the Docker image with the dev tag

I am using the dev tag now but I am getting and error without any source reference:

micronaut:
  security:
    enabled: true
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: pleasechangemesampletokentoteston
akhq.security:
  default-group: no-roles
  basic-auth:
    - username: feature_admin
      password: <ommited>
      groups:
      - admin
  groups:
    admin:
      - role: node-admin
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: topic-admin
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: connect-admin
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: registry-admin
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: group-read
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: connect-cluster-read
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]
      - role: ksqldb-admin
        patterns: [ "*feature-1-*" ]
        clusters: [ "feature-1" ]

akhq:
  connections:
    Feature-Internal-All:
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    feature-1:
      schema-registry:
        url: <ommited>
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    feature-2:
      schema-registry:
        url: <ommited>
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    feature-3:
      schema-registry:
        url: <ommited>
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

logs:

java.lang.NullPointerException: null                                                                                                                                                                               │
│     at org.akhq.controllers.AbstractController.lambda$buildUserBasedResourceFilters$5(AbstractController.java:96)                                                                                                  │
│     at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)                                                                                                                                     │
│     at java.base/java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)                                                                                                                                     │
│     at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)                                                                                                                         │
│     at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)                                                                                                                                        │
│     at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)                                                                                                                                 │
│     at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)                                                                                                                            │
│     at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)                                                                                                                                        │
│     at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)                                                                                                                                        │
│     at org.akhq.controllers.AbstractController.buildUserBasedResourceFilters(AbstractController.java:106)                                                                                                          │
│     at org.akhq.controllers.TopicController.list(TopicController.java:105)                                                                                                                                         │
│     at org.akhq.controllers.$TopicController$Definition$Exec.dispatch(Unknown Source)                                                                                                                              │
│     at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371)                                                                    │
│     at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594)                                                                                                                               │
│     at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303)                                                                                                                             │
│     at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111)                                                                                                                                             │
│     at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103)                                                                                                                          │
│     at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659)                                                                                                                       │
│     at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49)                                                                                                                           │
│     at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62)                                                                                                                         │
│     at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194)                                                                                                                  │
│     at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62)                                                                                               │
│     at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)                                                                                                                                                  │
│     at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)                                                                                                                                                  │
│     at io.micrometer.core.instrument.composite.CompositeTimer.recordCallable(CompositeTimer.java:129)                                                                                                              │
│     at io.micrometer.core.instrument.Timer.lambda$wrap$1(Timer.java:206)                                                                                                                                           │
│     at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53)                                                                                │
│     at java.base/java.util.concurrent.FutureTask.run(Unknown Source)                                                                                                                                               │
│     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)                                                                                                                                 │
│     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)                                                                                                                                │
│     at java.base/java.lang.Thread.run(Unknown Source)

Logs file: feature-12-deployment-akhq-7946c47cc5-7skgc-1690572261527824000.log

So as you can see above, I am trying to set 3 connections configuration. I am mainly trying to filter on each connection the topics corresponding to the feature namespace. Each topic has a structure like app-feature-<#>-service-, so what I and trying to achieve is to only see the topics with specific feature # per connection when selection the connection from the UI list. I am changing the admin group pattern an clusters attribute to test first with the topics prefixed with app-feature-1-service- expecting to see only feature-1 topics on each connection as this is the default group assigned but I am getting the error exposed.

Perhaps could you help me with a different structure suggestion to achieve this goal? Thank you in advance.

@josue-sanchez-wabtec you have to define the akhq.security.roles to make it works. If you copy/paste the one from the application.example.yml it should work fine :)

Thank yo so much @AlexisSouquiere @tchiotludo it is working for me now, I am sharing how I have the config now so it could be reference for others:

micronaut:
  security:
    enabled: true
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: <ommited>
akhq.security:
  roles:
    node-read:
      - resources: [ "NODE" ]
        actions: [ "READ", "READ_CONFIG" ]
    topic-read:
      - resources: [ "TOPIC", "TOPIC_DATA" ]
        actions: [ "READ" ]
      - resources: [ "TOPIC" ]
        actions: [ "READ_CONFIG" ]
    topic-admin:
      - resources: [ "TOPIC", "TOPIC_DATA" ]
        actions: [ "READ", "CREATE", "DELETE" ]
      - resources: [ "TOPIC" ]
        actions: [ "UPDATE", "READ_CONFIG", "ALTER_CONFIG" ]
    registry-read:
      - resources: [ "SCHEMA" ]
        actions: [ "READ" ]
    group-read:
      - resources: [ "CONSUMER_GROUP" ]
        actions: [ "READ" ]
  default-group: no-roles
  basic-auth:
    - username: feature_admin
      password: "<ommited>"
      groups:
      - admin
      - feature-1-group
      - feature-2-group
      - feature-3-group

  groups:
    admin:
      - role: topic-admin
        patterns: [ "topic-feature-us-internal-.*" ]
        clusters: [ "feature-internal-all" ]
      - role: group-read
    feature-1-group:
      - role: topic-read
        patterns: [ "topic-feature-us-internal-1-.*" ]
        clusters: [ "feature-1" ]
      - role: registry-read
      - role: group-read
    feature-2-group:
      - role: topic-read
        patterns: [ "topic-feature-us-internal-2-.*" ]
        clusters: [ "feature-2" ]
      - role: registry-read
      - role: group-read
    feature-3-group:
      - role: topic-read
        patterns: [ "topic-feature-us-internal-3-.*" ]
        clusters: [ "feature-3" ]
      - role: registry-read
      - role: group-read

akhq:
  connections:
    Feature-Internal-All:
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    Feature-1:
      schema-registry:
        url: https://<ommited>/schema-registry
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    Feature-2:
      schema-registry:
        url: https://<ommited>/schema-registry
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

    Feature-3:
      schema-registry:
        url: https://<ommited>/schema-registry
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' password='<ommited>';

This configuration is allowing me to have just one auth credential set to see all the filtering been applied on each connection/cluster.

Now I have another not expected behavior adding way more groups configurations and declaring them on the basic-auth for the same credential defined already. I need to map 100 connections as I handle 100 namespaces. For each of them I added the corresponding group config and the connection block too extending the config file I already have.

    feature-4-group:
      - role: topic-read
        patterns: [ "topic-feature-us-internal-4-.*" ]
        clusters: [ "feature-4" ]
      - role: registry-read
      - role: group-read
     ....
     feature-100-group:
      - role: topic-read
        patterns: [ "topic-feature-us-internal-100-.*" ]
        clusters: [ "feature-100" ]
      - role: registry-read
      - role: group-read

      .
      .
      .

 Feature-4:
      schema-registry:
        url: https://<ommited>/schema-registry
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>' 
       ....
Feature-100:
      schema-registry:
        url: https://<ommited>/schema-registry
        type: confluent
        basic-auth-username: <ommited>
        basic-auth-password: <ommited>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <ommited>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<ommited>'

After this I am not able to log in I am getting a User or Password Error:

Logs:

feature-12-deployment-akhq-b9887cd76-shrpb-1690991080055126000.log

I tried to limit the groups defined at:

From :

  basic-auth:
    - username: feature_admin
      password: "<ommited>"
      groups:
      - admin
      - feature-1-group
      - feature-2-group
      - feature-3-group
      - feature-4-group
      ...
      - feature-100-group

To only:

  basic-auth:
    - username: feature_admin
      password: "<ommited>"
      groups:
      - admin
      - feature-1-group
      - feature-2-group
      - feature-3-group
      - feature-4-group
      - feature-5-group
      - feature-6-group
      - feature-7-group
      - feature-8-group
      - feature-9-group
      - feature-10-group
      - feature-11-group

As I see until feature-11-group everything works well.

I am wondering perhaps, is there any limitation when declaring the auth groups that could be not allowing me to use this credentials any more? So here I have 12 groups declared, Is 12 a MAX limit here?

Thank you in advance for your help or a different approach you could suggest me.

There is no such constraint you can have 10 or 100 groups it’s the same. The only problem you will have now is with the JWT token stored in a cookie by default in AKHQ. A cookie has a 4ko limit so the JWT token too but it contains the groups you belong to. With 100 groups you are probably exceeding this limit

But AKHQ supports also auth with a bearer to bypass this limitation. Try to change micronaut.security.authentication: bearer. The token will be stored in the browser storage which doesn’t have this limit

If you still have the problem after you have to increase the default 8kb header size default limit from Netty because it means that 8kb isn’t enough too.

A3FDD86C-8F42-45BB-BFBA-DBBFFB6CC8A8

@AlexisSouquiere @tchiotludo Still an error after changing regarding the authentication:

micronaut:
  security:
    enabled: true
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: <ommited>

into

micronaut:
  security:
    enabled: true
    authentication: bearer
  server:
    netty:
      maxHeaderSize: 16384

Logs: feature-12-deployment-akhq-864984488c-vqz8r-1691006754965513000.log

Please keep the micronaut.security.token settings because we still create JWT token. We just send in to the front differently

Thank you so much @AlexisSouquiere and @tchiotludo for the help. I finally have what I was looking for. I share my current configuration to be used as reference for others that also whats to set a multi connection filtering setting.

I just want to mention in my case I need to set the maxHeaderSize to:

  server:
    netty:
      maxHeaderSize: 32768

Final configuration:

micronaut:
  security:
    enabled: true
    authentication: bearer
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: <omitted>
  server:
    netty:
      maxHeaderSize: 32768
akhq.security:
  roles:
    topic-admin:
      - resources: [ "TOPIC", "TOPIC_DATA" ]
        actions: [ "READ", "CREATE", "DELETE" ]
      - resources: [ "TOPIC" ]
        actions: [ "UPDATE", "READ_CONFIG", "ALTER_CONFIG" ]
    registry-admin:
      - resources: [ "SCHEMA" ]
        actions: [ "READ", "CREATE", "UPDATE", "DELETE", "DELETE_VERSION" ]
    group-read:
      - resources: [ "CONSUMER_GROUP" ]
        actions: [ "READ" ]
  default-group: no-roles
  basic-auth:
    - username: feature_admin
      password: "<omitted>"
      groups:
      - admin
      - feature-1-group
      - feature-2-group
      - feature-3-group
      - feature-4-group
      - feature-5-group
      - feature-6-group
      - feature-7-group
      - feature-8-group
      - feature-9-group
      - feature-10-group
      - feature-11-group
      - feature-12-group
      - feature-13-group
      - feature-14-group
      - feature-15-group
        ...
      - feature-100-group
  groups:
    admin:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-.*" ]
        clusters: [ "feature-internal-all", "shared-external" ]
      - role: group-read
    feature-1-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-1-.*" ]
        clusters: [ "feature-1" ]
      - role: registry-admin
      - role: group-read
    feature-2-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-2-.*" ]
        clusters: [ "feature-2" ]
      - role: registry-admin
      - role: group-read
    feature-3-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-3-.*" ]
        clusters: [ "feature-3" ]
      - role: registry-admin
      - role: group-read
    feature-4-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-4-.*" ]
        clusters: [ "feature-4" ]
      - role: registry-admin
      - role: group-read
    feature-5-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-5-.*" ]
        clusters: [ "feature-5" ]
      - role: registry-admin
      - role: group-read
    feature-6-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-6-.*" ]
        clusters: [ "feature-6" ]
      - role: registry-admin
      - role: group-read
    feature-7-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-7-.*" ]
        clusters: [ "feature-7" ]
      - role: registry-admin
      - role: group-read
    feature-8-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-8-.*" ]
        clusters: [ "feature-8" ]
      - role: registry-admin
      - role: group-read
    feature-9-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-9-.*" ]
        clusters: [ "feature-9" ]
      - role: registry-admin
      - role: group-read
    feature-10-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-10-.*" ]
        clusters: [ "feature-10" ]
      - role: registry-admin
      - role: group-read
    feature-11-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-11-.*" ]
        clusters: [ "feature-11" ]
      - role: registry-admin
      - role: group-read
    feature-12-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-12-.*" ]
        clusters: [ "feature-12" ]
      - role: registry-admin
      - role: group-read
    feature-13-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-13-.*" ]
        clusters: [ "feature-13" ]
      - role: registry-admin
      - role: group-read
    feature-14-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-14-.*" ]
        clusters: [ "feature-14" ]
      - role: registry-admin
      - role: group-read
    feature-15-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-15-.*" ]
        clusters: [ "feature-15" ]
      - role: registry-admin
      - role: group-read
    ...
    feature-100-group:
      - role: topic-admin
        patterns: [ "pds2-feature-us-internal-100-.*" ]
        clusters: [ "feature-100" ]
      - role: registry-admin
      - role: group-read

akhq:
  connections:
    Feature-Internal-All:
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-1:
      schema-registry:
        url: https://feature-1.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-2:
      schema-registry:
        url: https://feature-2.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-3:
      schema-registry:
        url: https://feature-3.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-4:
      schema-registry:
        url: https://feature-4.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-5:
      schema-registry:
        url: https://feature-5.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-6:
      schema-registry:
        url: https://feature-6.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-7:
      schema-registry:
        url: https://feature-7.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-8:
      schema-registry:
        url: https://feature-8.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-9:
      schema-registry:
        url: https://feature-9.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-10:
      schema-registry:
        url: https://feature-10.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-11:
      schema-registry:
        url: https://feature-11.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-12:
      schema-registry:
        url: https://feature-12.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-13:
      schema-registry:
        url: https://feature-13.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-14:
      schema-registry:
        url: https://feature-14.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    Feature-15:
      schema-registry:
        url: https://feature-15.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

    ...

    Feature-100:
      schema-registry:
        url: https://feature-100.<omitted>/schema-registry
        type: confluent
        basic-auth-username: <omitted>
        basic-auth-password: <omitted>
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';
    Shared-External:
      ui-options:
        topic-data:
          sort: NEWEST
      properties:
        bootstrap.servers: <omitted>:9092
        security.protocol: SASL_SSL
        sasl.mechanism: PLAIN
        sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username='<omitted>' password='<omitted>';

@AlexisSouquiere @tchiotludo Additionally, is there any way to hide the clusters not added as part of the group configuration for a specific auth user?

I have the configiguration as:

  basic-auth:
    - username: feature_admin
      password: "<omitted>"
      groups:
      - admin
    - username: feature_1
      password: "<omitted>"
      groups:
      - feature-1-group
    - username: feature_2
      password: "<omitted>"
      groups:
      - feature-2-group
    - username: feature_3
      password: "<omitted>"
      groups:
      - feature-3-group

groups:
    admin:
      - role: node-admin
      - role: topic-admin
        patterns: [ "topic-feature-us-internal.*" ]
        clusters: [ "feature-internal-all", "shared-external" ]
      - role: group-read
    feature-1-group:
      - role: node-admin
      - role: topic-admin
        patterns: [ "topic-feature-us-internal-1-.*" ]
        clusters: [ "feature-1" ]
      - role: registry-admin
      - role: group-read
    feature-2-group:
      - role: node-admin
      - role: topic-admin
        patterns: [ "topic-feature-us-internal-2-.*" ]
        clusters: [ "feature-2" ]
      - role: registry-admin
      - role: group-read
    feature-3-group:
      - role: node-admin
      - role: topic-admin
        patterns: [ "topic-feature-us-internal-3-.*" ]
        clusters: [ "feature-3" ]
      - role: registry-admin
      - role: group-read

Or is there any way to disable the redirection to the first connection mapped ? I am always being redirected to the first connection "...ui/feature-1/topic" even when using other credentials.

Clusters list in the sidebar is already filtered based on the user permission. The issue you have here is because the value of the clusters attribute should be an array of regexes. By giving clusters: [ "feature-1" ] all the clusters that contains feature-1 will match. So does feature-12, feature-13, etc.

I realize that it's something I didn't explained in the doc (I need to update it) but if you give clusters: [ "^\\Qfeature-1\\E$"], the clusters list will only show feature-1 cluster

@AlexisSouquiere Well I am now testing with only one group defined and one auth credentials. This groups should match for the cluster specified:

  basic-auth:
    - username: feature_admin
      password: "<omitted>"
      groups:
      - admin
  groups:
    admin:
      - role: node-admin
      - role: topic-admin
        patterns: [ ".*" ]
        clusters: [ "^\\QFeature-Internal-All\\E$" ]
      - role: group-read

I even changed the clusters attribute to>

clusters: [ "^Feature-Internal-All$" ]

and

clusters: [ "Feature-Internal-All" ]

just in case but still not working.

So due to this when log in it is always redirecting be to "...ui/feature-1/topic" as I have all the clusters defined. I did not remove anything from connections section.

Sorry I read your configuration too quickly. By not specifying the same clusters restrictions on the node-admin, registry-admin and group-read, it means that you have an "*" restrictions for these roles. And because you have at least one permissions on all the clusters, you can see them in the list

It worked. Adding clusters and patterns to all roles defined for each group helped to filter the list.

I have now a user experience strange behavior that I would like to expose and see perhaps if this is related to my configuration: This is what I am doing:

1.- Log in as admin: 2.- I can see the filter applied 3.- I select any topic so I can see the url path changing 4.- Close the session

5.- Log in with other user, and here is where I noticed there is not any redirection to at least /new-connection/topic. So when trying to access with other user I am not able to logging as it seems to redirect me to the last session path opened.

The session closes

Shouldnt I be redirected to the /new-connection/topic path when using other user account?

@josue-sanchez-wabtec seems to be another issue, not a simple one to correct. Please open another issue please. Side note: not a lot of possibility: don't keep the current url when unlogged

tchiotludo / akhq

topics.regex configuration error #1537