KAFKAHQ issue "Internal Server Error

[proprama]

Dear All,

Please help me in resolving the issue ,I am getting an error when I am trying to access the UI

{"message":"Internal Server Error: Couldn't find any clusters on your configuration file, please ensure that the configuration file is loaded correctly"}.

The configuration file is as below :- `micronaut: security: enabled: true endpoints: login: enabled: false logout: enabled: false akhq: server: access-log: # Access log configuration (optional) enabled: true # true by default name: org.akhq.log.access # Logger name format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

default kafka properties for each clients, available for admin / producer / consumer (optional)

clients-defaults: consumer: properties: isolation.level: read_committed

list of kafka cluster available for akhq

connections: my-cluster-plain-text: # url friendly name for the cluster (letter, number, _, -, ... dot are not allowed here) properties: # standard kafka properties (optional) bootstrap.servers: "XXXXXXXX.com:9092"

pagination: page-size: 25 # number of elements per page (default : 25) threads: 16 # Number of parallel threads to resolve page

Topic list display options (optional)

topic: retention: 172800000 # default retention period when creating topic partition: 3 # default number of partition when creating topic replication: 3 # default number of replicas when creating topic default-view: HIDE_INTERNAL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM) internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)

• "^_.*$"
• "^.*_schemas$"
• "^.*connect-config$"
• "^.*connect-offsets$1"
• "^.*connect-status$" stream-regexps: # list of regexp to be considered as internal stream topic
• "^.*-changelog$"
• "^.*-repartition$"

• "^.*-rekey$" skip-consumer-groups: false # Skip loading consumer group information when showing topics

  Topic display data options (optional)

  topic-data:

  sort: OLDEST # default sort order (OLDEST, NEWEST) (default: OLDEST)

  size: 50 # max record per page (default: 50)

  poll-timeout: 1000 # The time, in milliseconds, spent waiting in poll if data is not available in the buffer.

  Auth & Roles (optional)

  security: security: default-group: admin # Default groups for all the user even unlogged user

  Groups definition

  groups:

  • name: admin # Group name roles: # roles for the group
  • topic/read
  • topic/insert
  • topic/delete
  • topic/config/update
  • node/read
  • node/config/update
  • topic/data/read
  • topic/data/insert
  • topic/data/delete
  • group/read
  • group/delete
  • group/offsets/update
  • registry/read
  • registry/insert
  • registry/update
  • registry/delete
  • registry/version/delete
  • acls/read
  • connect/read
  • connect/insert
  • connect/update
  • connect/delete
  • connect/state/update attributes:

    Regexp to filter topic available for group

    topics-filter-regexp: "test.*"

  • name: topic-reader # Other group roles:
  • topic/read

    attributes:

    topics-filter-regexp: "test\.reader.*"

    Basic auth configuration

    basic-auth: AXXXX password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7 groups: # Groups for the user

  • admin
  • topic-reader default-group: reader # Default groups for all the user even unlogged user

    Groups definition

    groups:

    - name: admin # Group name

    roles: # roles for the group

    - topic/read

    - node/read

    - group/read

    attributes:

    Regexp to filter topic available for group

    topics-filter-regexp: "test.*"`

I am trying to login to the UI using AXXXXXX.

FYI I have used the below command to generate the password echo -n "password" | sha256sum which I have used in the configuration file .

I am running it as below docker run -d \ -p 8080:8080 \ -v /tmp/application.yml:/app/application.yml \ tchiotludo/akhq.

Kindly help

Thanks Prosenjit

[tchiotludo]

Please format the yaml it's unreadable and keep only your parameters and remove all default one (only things you have edited).

Thanks

[proprama]

Hi Thanks for the update ,Please find the the changes that I have done , to mark my changes I have added <---My Addition-->
`micronaut: security: enabled: true endpoints: login: enabled: false logout: enabled: false akhq: server: access-log: # Access log configuration (optional) enabled: true # true by default name: org.akhq.log.access # Logger name format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

default kafka properties for each clients, available for admin / producer / consumer (optional)

clients-defaults: consumer: properties: isolation.level: read_committed

list of kafka cluster available for akhq

connections: my-cluster-plain-text: # url friendly name for the cluster (letter, number, _, -, ... dot are not allowed here) properties: # standard kafka properties (optional) <--------My Addition ---- > bootstrap.servers: "XXXXXXXX.com:9092" <--------Addition END ---- >

pagination: page-size: 25 # number of elements per page (default : 25) threads: 16 # Number of parallel threads to resolve page

Topic list display options (optional)

topic: retention: 172800000 # default retention period when creating topic partition: 3 # default number of partition when creating topic replication: 3 # default number of replicas when creating topic default-view: HIDE_INTERNAL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM) internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)

• "^_.*$"
• "^.*_schemas$"
• "^.*connect-config$"
• "^.*connect-offsets$1"
• "^.*connect-status$" stream-regexps: # list of regexp to be considered as internal stream topic
• "^.*-changelog$"
• "^.*-repartition$"
• "^.*-rekey$" skip-consumer-groups: false # Skip loading consumer group information when showing topics

Auth & Roles (optional)

security: security: default-group: admin # Default groups for all the user even unlogged user

Groups definition

groups:

• name: admin # Group name roles: # roles for the group
• topic/read
• topic/insert
• topic/delete
• topic/config/update
• node/read
• node/config/update
• topic/data/read
• topic/data/insert
• topic/data/delete
• group/read
• group/delete
• group/offsets/update
• registry/read
• registry/insert
• registry/update
• registry/delete
• registry/version/delete
• acls/read
• connect/read
• connect/insert
• connect/update
• connect/delete
• connect/state/update attributes:

  Regexp to filter topic available for group

  topics-filter-regexp: "test.*"

• name: topic-reader # Other group roles:
• topic/read <---My Addition--> basic-auth: AXXXXXX: password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7 <---Till this --> groups: # Groups for the user
• admin
• topic-reader default-group: reader `

Kindly let me know if this is fine

[tchiotludo]

so remove all the others things and just keep this additions and send me the final yaml attach or formatted with with triple `

[proprama]

Hi,

# Auth & Roles (optional)
  security:
 security:
default-group: admin # Default groups for all the user even unlogged user
# Groups definition
groups:
- name: admin # Group name
roles: # roles for the group
- topic/read
- topic/insert
- topic/delete
- topic/config/update
- node/read
- node/config/update
- topic/data/read
- topic/data/insert
- topic/data/delete
- group/read
- group/delete
- group/offsets/update
- registry/read
- registry/insert
- registry/update
- registry/delete
- registry/version/delete
- acls/read
- connect/read
- connect/insert
- connect/update
- connect/delete
- connect/state/update
attributes:
# Regexp to filter topic available for group
topics-filter-regexp: "test.*"
- name: topic-reader # Other group
roles:
- topic/read
<---My Addition-->
basic-auth:
AXXXXXXX:
password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7
<---Till this -->
groups: # Groups for the user
- admin
- topic-reader
    default-group: reader 

[tchiotludo]

it's a Yaml !! You don't have any indentation, so this don't work

[proprama]

Is there any YAML editor ,where I can check the indentation ,can u please let me know or a sample file with proper Indentation for adding basic-auth

[tchiotludo]

https://codebeautify.org/yaml-validator with a validator. Just paste the example config remove all the default and just keep the things you changed

[proprama]

Thanks so much. I will keep you posted and let you know the outcome, please don't close the ticket

[proprama]

Hi,

I use the YAML validator and I ran the code

micronaut:
  security:
    enabled: true
    endpoints:
      login:
        enabled: false
      logout:
        enabled: false
akhq:
  server:
    access-log: # Access log configuration (optional)
      enabled: true # true by default
      name: org.akhq.log.access # Logger name
      format: "[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User: {}]" # Logger format

  # default kafka properties for each clients, available for admin / producer / consumer (optional)
  clients-defaults:
    consumer:
      properties:
        isolation.level: read_committed

  # list of kafka cluster available for akhq
  connections:
    my-cluster-plain-text: # url friendly name for the cluster (letter, number, _, -, ... dot are not allowed here)
      properties: # standard kafka properties (optional)
        bootstrap.servers: "xxxxxx:9092"

  pagination:
    page-size: 25 # number of elements per page (default : 25)
    threads: 16 # Number of parallel threads to resolve page

  # Topic list display options (optional)
  topic:
    retention: 172800000 # default retention period when creating topic
    partition: 3 #  default number of partition when creating topic
    replication: 3 # default number of replicas when creating topic
    default-view: HIDE_INTERNAL # default list view (ALL, HIDE_INTERNAL, HIDE_INTERNAL_STREAM, HIDE_STREAM)
    internal-regexps: # list of regexp to be considered as internal (internal topic can't be deleted or updated)
      - "^_.*$"
      - "^.*_schemas$"
      - "^.*connect-config$"
      - "^.*connect-offsets$1"
      - "^.*connect-status$"
    stream-regexps: # list of regexp to be considered as internal stream topic
      - "^.*-changelog$"
      - "^.*-repartition$"
      - "^.*-rekey$"
    skip-consumer-groups: false # Skip loading consumer group information when showing topics

  # Topic display data options (optional)
#  topic-data:
#    sort: OLDEST # default sort order (OLDEST, NEWEST) (default: OLDEST)
#    size: 50 # max record per page (default: 50)
#    poll-timeout: 1000 # The time, in milliseconds, spent waiting in poll if data is not available in the buffer.
# Auth & Roles (optional)
# Basic auth configuration
basic-auth:
user: Axxxxxxxx
password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7
groups: # Groups for the user
- admin
- topic-reader
    default-group: reader # Default groups for all the user even unlogged user
    # Groups definition
#    groups:
#      - name: admin # Group name
#        roles:  # roles for the group
#          - topic/read
#          - node/read
#          - group/read
#        attributes:
          # Regexp to filter topic available for group
#          topics-filter-regexp: "test.*

It still gives me the error 

{"message":"Internal Server Error: Couldn't find any clusters on your configuration file, please ensure that the configuration file is loaded correctly"}

PLease help

[tchiotludo]

Let's start with a simple files :

akhq:
  connections:
    bestbuy-na: 
      properties: 
        bootstrap.servers: "dtl01s8jmq01c.na.bestbuy.com:9092"

will be sufficient to go on.

If it's don't work, it's the files that is not detected. How do you launch akhq ?

[proprama]

Hi I am lqunching thru docker

akhq:
  connections:
    bestby-na: 
      properties: 
        bootstrap.servers: "dXXXXXXX:9092"

This piece of code works ,it only when I apply the AUTH or the

basic-auth:
user: Axxxxxxx
password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7

it gives me the error

{"message":"Internal Server Error: Couldn't find any clusters on your configuration file, please ensure that the configuration file is loaded correctly"}

please help

[proprama]

HI,

My requirement is that I login in the UI thru the username AXXXXXXXX and password ,that the reason I am adding the basic-auth

[tchiotludo]

micronaut:
  security:
    enabled: true

akhq:
  connections:
    bestbuy-na: 
      properties: 
        bootstrap.servers: "dtl01s8jmq01c.na.bestbuy.com:9092"

  security:
    default-group: reader

    basic-auth:
      - username: A1509893
        password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7
        groups:
          - admin

must do the trick what last version 0.16.0 (published yesterday)

[proprama]

Hi,

Thanks for the update will update you in a moment .

[proprama]

Hi,

I ran into same issue , after the suggested changes .I am running as below

sudo docker run -d  -p 8080:8080  -v /tmp/application_akhq.yml:/app/application_akhq.yml  tchiotludo/akhq
8c7716e86c5c9d62a3cd5c60aa31f80efdacb0c65e73560e80268054f51b6668
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                    NAMES
8c7716e86c5c        tchiotludo/akhq     "docker-entrypoint..."   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp   sad_goldstine

the content of the application_akhq.yml is the exact ,as suggested by you.

it gives me the error

{"message":"Internal Server Error: Couldn't find any clusters on your configuration file, please ensure that the configuration file is loaded correctly"}

please help

[tchiotludo]

/app/application_akhq.yml will not works !

it must be /app/application.yml or change the env vars MICRONAUT_CONFIG_FILES, see here && here

[proprama]

Hi ,

Thanks for the update ,I ran as guided

sudo docker run -d  -p 8080:8080  -v /tmp/application.yml:/app/application.yml  tchiotludo/akhq

I also checked the docker logs ,I could only see info and no errors in it .

However my GUI was up and asking for username and password ,when I enter the user name Axxxxxx and password which is correct ,I am unable to login to the KAFKAHQ UI .

Kindly let me know as how to debug or what has gone wrong

Thanks

[tchiotludo]

mostly it will be password that is wrong encoded:

try with this with login: user and password: pass

    basic-auth:
      - username: user
        password: d74ff0ee8da3b9806b18c877dbf29bbde50b5bd8e4dad7a3a725000feb82e8f1
        groups:
          - admin

[proprama]

Ok ,

I will try with this username: user and password pass ,Thanks will update u

[proprama]

Hi ,

You may please close the issue . Thanks for helping

[proprama]

The issue got resolved so closing the ticket

[proprama]

micronaut:
  security:
    enabled: true

akhq:
  connections:
    buy-na: 
      properties: 
        bootstrap.servers: "XXXXXXXXXX:9092"

  security:
    default-group: reader

    basic-auth:
      - username: AXXXXX
        password: f01d2e7ec1ed1d9ac16e3f11fcd61d5a9c1e8293d31089f275c3ac15fe0667d7
        groups:
          - admin

must do the trick what last version 0.16.0 (published yesterday)

[tchiotludo]

don't forgot this one : https://github.com/tchiotludo/akhq/issues/161 to thanks for support :smile: