search

bitnami / containers

Bitnami container images
https://bitnami.com
Other
3.26k stars 4.75k forks source link

Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/ error #3588

Closed patiltussh closed 1 year ago

patiltussh commented 2 years ago

Name and Version

bitnami/kafka:3.2.1

What steps will reproduce the bug?

  1. Execute the below command 
    docker run --rm --net=host \
--ulimit nofile="65536" \
-v /root/kafka:/bitnami/kafka \
-p 9092:9092 \
-p 9093:9093 \
-p 9999:9999 \
-p 8083:8083 \
-p 8888:8888 \
-p 8889:8889 \
-e KAFKA_ZOOKEEPER_PROTOCOL=SSL \
-e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,SSL://:9093 \
-e KAFKA_ZOOKEEPER_TLS_KEYSTORE_PASSWORD=<zookeeper keystore password> \
-e KAFKA_ZOOKEEPER_TLS_TRUSTSTORE_PASSWORD=<zookeeper truststore password> \
-e KAFKA_ZOOKEEPER_TLS_VERIFY_HOSTNAME=false \
-e KAFKA_ZOOKEEPER_TLS_TYPE=JKS \
-e KAFKA_CFG_ZOOKEEPER_CONNECT="zookeeper-hostname:2281" \
-e KAFKA_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.rmi.port=9999 -Djava.rmi.server.hostname=<IP Address of host> -Djava.net.preferIPv4Stack=true" \
--name=kafka \
bitnami/kafka:3.2.1
  2. Execution of the above command is giving the below error. Because of this error, kafka container is not running 
    
WARNING: Published ports are discarded when using host network mode
kafka 03:29:03.30
kafka 03:29:03.30 Welcome to the Bitnami kafka container
kafka 03:29:03.31 Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 03:29:03.31 Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 03:29:03.31
kafka 03:29:03.32 INFO  ==> ** Starting Kafka setup **
kafka 03:29:03.53 INFO  ==> Initializing Kafka...
kafka 03:29:03.61 INFO  ==> Copying truststore /bitnami/kafka/config/certs/kafka.truststore.jks to /opt/bitnami/kafka/config/certs
kafka 03:29:03.62 INFO  ==> Copying truststore /bitnami/kafka/config/certs/zookeeper.truststore.jks to /opt/bitnami/kafka/config/certs
kafka 03:29:03.63 INFO  ==> No injected configuration files found, creating default config files

kafka 03:29:03.99 INFO ==> Kafka setup finished! kafka 03:29:04.02 INFO ==> Starting Kafka [2022-08-19 03:29:06,441] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [2022-08-19 03:29:07,356] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util) [2022-08-19 03:29:07,501] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler) [2022-08-19 03:29:07,510] INFO starting (kafka.server.KafkaServer) [2022-08-19 03:29:07,511] INFO Connecting to zookeeper on zookeeper-host:2281 (kafka.server.KafkaServer) [2022-08-19 03:29:07,539] INFO [ZooKeeperClient Kafka server] Initializing a new session to zookeeper-host:2281. (kafka.zookeeper.ZooKeeperClient) [2022-08-19 03:29:07,552] INFO Client environment:zookeeper.version=3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a, built on 04/08/2021 16:35 GMT (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:host.name=ub51177.starbucks.net (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:java.version=11.0.15.1 (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:java.vendor=BellSoft (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:java.home=/opt/bitnami/java (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:java.class.path=/opt/bitnami/kafka/bin/../libs/activation-1.1.1.jar:/opt/bitnami/kafka/bin/../libs/aopalliance-repackaged-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/bitnami/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/bitnami/kafka/bin/../libs/commons-cli-1.4.jar:/opt/bitnami/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/bitnami/kafka/bin/../libs/connect-api-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-basic-auth-extension-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-json-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-mirror-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-mirror-client-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-runtime-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/connect-transforms-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/hk2-api-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/hk2-locator-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/hk2-utils-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/jackson-annotations-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-core-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-databind-2.12.6.1.jar:/opt/bitnami/kafka/bin/../libs/jackson-dataformat-csv-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-datatype-jdk8-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-jaxrs-base-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-jaxrs-json-provider-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-module-jaxb-annotations-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jackson-module-scala_2.12-2.12.6.jar:/opt/bitnami/kafka/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/bitnami/kafka/bin/../libs/jakarta.annotation-api-1.3.5.jar:/opt/bitnami/kafka/bin/../libs/jakarta.inject-2.6.1.jar:/opt/bitnami/kafka/bin/../libs/jakarta.validation-api-2.0.2.jar:/opt/bitnami/kafka/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/opt/bitnami/kafka/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/bitnami/kafka/bin/../libs/javassist-3.27.0-GA.jar:/opt/bitnami/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/bitnami/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/bitnami/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/bitnami/kafka/bin/../libs/jersey-client-2.34.jar:/opt/bitnami/kafka/bin/../libs/jersey-common-2.34.jar:/opt/bitnami/kafka/bin/../libs/jersey-container-servlet-2.34.jar:/opt/bitnami/kafka/bin/../libs/jersey-container-servlet-core-2.34.jar:/opt/bitnami/kafka/bin/../libs/jersey-hk2-2.34.jar:/opt/bitnami/kafka/bin/../libs/jersey-server-2.34.jar:/opt/bitnami/kafka/bin/../libs/jetty-client-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-continuation-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-http-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-io-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-security-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-server-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-servlet-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-servlets-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-util-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jetty-util-ajax-9.4.44.v20210927.jar:/opt/bitnami/kafka/bin/../libs/jline-3.21.0.jar:/opt/bitnami/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/bitnami/kafka/bin/../libs/jose4j-0.7.9.jar:/opt/bitnami/kafka/bin/../libs/kafka-clients-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-log4j-appender-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-metadata-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-raft-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-server-common-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-shell-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-storage-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-storage-api-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-examples-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-scala_2.12-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-streams-test-utils-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka-tools-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/kafka_2.12-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/lz4-java-1.8.0.jar:/opt/bitnami/kafka/bin/../libs/maven-artifact-3.8.4.jar:/opt/bitnami/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/bitnami/kafka/bin/../libs/metrics-core-4.1.12.1.jar:/opt/bitnami/kafka/bin/../libs/netty-buffer-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-codec-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-common-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-handler-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-resolver-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-tcnative-classes-2.0.46.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-classes-epoll-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-native-epoll-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/netty-transport-native-unix-common-4.1.73.Final.jar:/opt/bitnami/kafka/bin/../libs/osgi-resource-locator-1.0.3.jar:/opt/bitnami/kafka/bin/../libs/paranamer-2.8.jar:/opt/bitnami/kafka/bin/../libs/plexus-utils-3.3.0.jar:/opt/bitnami/kafka/bin/../libs/reflections-0.9.12.jar:/opt/bitnami/kafka/bin/../libs/reload4j-1.2.19.jar:/opt/bitnami/kafka/bin/../libs/rocksdbjni-6.29.4.1.jar:/opt/bitnami/kafka/bin/../libs/scala-collection-compat_2.12-2.6.0.jar:/opt/bitnami/kafka/bin/../libs/scala-java8-compat_2.12-1.0.2.jar:/opt/bitnami/kafka/bin/../libs/scala-library-2.12.15.jar:/opt/bitnami/kafka/bin/../libs/scala-logging_2.12-3.9.4.jar:/opt/bitnami/kafka/bin/../libs/scala-reflect-2.12.15.jar:/opt/bitnami/kafka/bin/../libs/slf4j-api-1.7.36.jar:/opt/bitnami/kafka/bin/../libs/slf4j-reload4j-1.7.36.jar:/opt/bitnami/kafka/bin/../libs/snappy-java-1.1.8.4.jar:/opt/bitnami/kafka/bin/../libs/trogdor-3.2.1.jar:/opt/bitnami/kafka/bin/../libs/zookeeper-3.6.3.jar:/opt/bitnami/kafka/bin/../libs/zookeeper-jute-3.6.3.jar:/opt/bitnami/kafka/bin/../libs/zstd-jni-1.5.2-1.jar (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,552] INFO Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:java.compiler= (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.version=4.15.0-50-generic (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:user.name=? (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:user.home=? (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:user.dir=/ (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.memory.free=1008MB (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.memory.max=1024MB (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,553] INFO Client environment:os.memory.total=1024MB (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,563] INFO Initiating client connection, connectString=zookeeper-host:2281 sessionTimeout=18000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@44e3a2b2 (org.apache.zookeeper.ZooKeeper) [2022-08-19 03:29:07,713] INFO jute.maxbuffer value is 4194304 Bytes (org.apache.zookeeper.ClientCnxnSocket) [2022-08-19 03:29:07,721] INFO zookeeper.request.timeout value is 0. feature enabled=false (org.apache.zookeeper.ClientCnxn) [2022-08-19 03:29:07,746] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient) [2022-08-19 03:29:07,760] INFO Opening socket connection to server zookeeper-host/10.12.205.73:2281. (org.apache.zookeeper.ClientCnxn) [2022-08-19 03:29:08,176] WARN Failed to initialize a channel. Closing: [id: 0x46a54dfb] (io.netty.channel.ChannelInitializer) org.apache.zookeeper.common.X509Exception$SSLContextException: Failed to create TrustManager at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:371) at org.apache.zookeeper.common.X509Util.createSSLContextAndOptions(X509Util.java:328) at org.apache.zookeeper.common.X509Util.createSSLContext(X509Util.java:256) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initSSL(ClientCnxnSocketNetty.java:448) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:438) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:422) at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938) at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609) at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46) at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463) at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115) at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650) at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514) at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429) at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/ at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:512) at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:369) ... 23 more Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/ at org.apache.zookeeper.common.KeyStoreFileType.fromFilename(KeyStoreFileType.java:90) at org.apache.zookeeper.common.KeyStoreFileType.fromPropertyValueOrFileName(KeyStoreFileType.java:112) at org.apache.zookeeper.common.X509Util.loadTrustStore(X509Util.java:407) at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:488) ... 24 more [2022-08-19 03:29:08,204] WARN future isn't success. (org.apache.zookeeper.ClientCnxnSocketNetty) io.netty.channel.StacklessClosedChannelException at io.netty.channel.AbstractChannel$AbstractUnsafe.ensureOpen(ChannelPromise)(Unknown Source) [2022-08-19 03:29:09,210] INFO Opening socket connection to server zookeeper-host/10.12.205.73:2281. (org.apache.zookeeper.ClientCnxn) [2022-08-19 03:29:09,213] WARN Failed to initialize a channel. Closing: [id: 0xbc480eaf] (io.netty.channel.ChannelInitializer) org.apache.zookeeper.common.X509Exception$SSLContextException: Failed to create TrustManager at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:371) at org.apache.zookeeper.common.X509Util.createSSLContextAndOptions(X509Util.java:328) at org.apache.zookeeper.common.X509Util.createSSLContext(X509Util.java:256) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initSSL(ClientCnxnSocketNetty.java:448) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:438) at org.apache.zookeeper.ClientCnxnSocketNetty$ZKClientPipelineFactory.initChannel(ClientCnxnSocketNetty.java:422) at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:129) at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:112) at io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded(AbstractChannelHandlerContext.java:938) at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:609) at io.netty.channel.DefaultChannelPipeline.access$100(DefaultChannelPipeline.java:46) at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1463) at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1115) at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:650) at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:514) at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:429) at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:486) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/ at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:512) at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:369) ... 23 more Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/ at org.apache.zookeeper.common.KeyStoreFileType.fromFilename(KeyStoreFileType.java:90) at org.apache.zookeeper.common.KeyStoreFileType.fromPropertyValueOrFileName(KeyStoreFileType.java:112) at org.apache.zookeeper.common.X509Util.loadTrustStore(X509Util.java:407) at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:488) ... 24 more [2022-08-19 03:29:09,217] WARN future isn't success. (org.apache.zookeeper.ClientCnxnSocketNetty) io.netty.channel.StacklessClosedChannelException at io.netty.channel.AbstractChannel$AbstractUnsafe.ensureOpen(ChannelPromise)(Unknown Source) ^C[2022-08-19 03:29:09,420] INFO Terminating process due to signal SIGINT (org.apache.kafka.common.utils.LoggingSignalHandler) [2022-08-19 03:29:09,422] INFO shutting down (kafka.server.KafkaServer) [2022-08-19 03:29:09,434] ERROR Fatal error during KafkaServer shutdown. (kafka.server.KafkaServer) java.lang.IllegalStateException: Kafka server is still starting up, cannot shut down! at kafka.server.KafkaServer.shutdown(KafkaServer.scala:705) at kafka.Kafka$.$anonfun$main$3(Kafka.scala:100) at kafka.utils.Exit$.$anonfun$addShutdownHook$1(Exit.scala:38) at java.base/java.lang.Thread.run(Thread.java:829) [2022-08-19 03:29:09,434] ERROR Halting Kafka. (kafka.Kafka$)


### What is the expected behavior?

The Kafka container should run without any issues. 

### Additional information

We are not observing this issue with Docker images with version bitnami/kafka:2.7.0 and less.
We are observing this issue with Docker images with version bitnami/kafka:2.8.0 and greater.
Mauraza commented 2 years ago

Hi @patiltussh,

It seems that Kafka fails because:

Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/

Did you mount these files? More information in the README.md

patiltussh commented 2 years ago

Hi @Mauraza

I have mounted files like below - -v /root/kafka:/bitnami/kafka

And content of /root/kafka directory looks like below -

root@kafkanode:~/kafka/config/certs# pwd
/root/kafka/config/certs
root@kafkanode:~/kafka/config/certs# ls -ltra
total 24
-rwxr-xr-x 1 automation automation 1062 Aug 18 18:28 kafka.truststore.jks
-rwxr-xr-x 1 automation automation 4267 Aug 18 18:28 kafka.keystore.jks
-rwxr-xr-x 1 automation automation 4276 Aug 18 18:44 zookeeper.keystore.jks
-rwxr-xr-x 1 automation automation 1062 Aug 18 18:44 zookeeper.truststore.jks
drwxr-xr-x 2 automation automation  122 Aug 18 18:44 .
drwxr-xr-x 3 automation automation   44 Aug 19 18:01 ..

Also in the kafka logs, I can see below message which indicate that kafka is copying files to /opt/bitnami/kafka/config/certs directory - 

kafka 03:29:03.61 INFO  ==> Copying truststore /bitnami/kafka/config/certs/kafka.truststore.jks to /opt/bitnami/kafka/config/certs
kafka 03:29:03.62 INFO  ==> Copying truststore /bitnami/kafka/config/certs/zookeeper.truststore.jks to /opt/bitnami/kafka/config/certs
Mauraza commented 2 years ago

Hi @patiltussh,

yes, that is 'truststore', the files don't found are the 'keystore'. Could you check if the keystore files are in /opt/bitnami/kafka/config/certs?

patiltussh commented 2 years ago

Hi @Mauraza

Yes. keystore files are present in /opt/bitnami/kafka/config/certs as below -

/var/lib/docker/overlay2/5d8ba25becc1facaf491a797737a37fe6431bdf605b6975cd2182884aceaca62/diff/opt/bitnami/kafka/config/certs/kafka.keystore.jks
/var/lib/docker/overlay2/5d8ba25becc1facaf491a797737a37fe6431bdf605b6975cd2182884aceaca62/diff/opt/bitnami/kafka/config/certs/zookeeper.keystore.jks
Mauraza commented 2 years ago

Hi @patiltussh,

Could you try to mount the files like appear in the documentation?

    volumes:
      - './kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro'
patiltussh commented 2 years ago

Hi @Mauraza

I tried by mounting files as below -

-v /root/kafka/config/certs/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro \
-v /root/kafka/config/certs/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks:ro \
-v /root/kafka/config/certs/zookeeper.keystore.jks:/opt/bitnami/kafka/config/certs/zookeeper.keystore.jks:ro \
-v /root/kafka/config/certs/zookeeper.truststore.jks:/opt/bitnami/kafka/config/certs/zookeeper.truststore.jks:ro \

Complete command -

/usr/bin/docker run --rm --net=host \
--ulimit nofile="65536" \
-v /root/kafka/config/certs/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro \
-v /root/kafka/config/certs/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks:ro \
-v /root/kafka/config/certs/zookeeper.keystore.jks:/opt/bitnami/kafka/config/certs/zookeeper.keystore.jks:ro \
-v /root/kafka/config/certs/zookeeper.truststore.jks:/opt/bitnami/kafka/config/certs/zookeeper.truststore.jks:ro \
-p 9092:9092 \
-p 9093:9093 \
-p 9999:9999 \
-p 8083:8083 \
-p 8888:8888 \
-p 8889:8889 \
-e KAFKA_ZOOKEEPER_PROTOCOL=SSL \
-e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,SSL://:9093 \
-e KAFKA_ZOOKEEPER_TLS_KEYSTORE_PASSWORD=test12 \
-e KAFKA_ZOOKEEPER_TLS_TRUSTSTORE_PASSWORD=test12 \
-e KAFKA_ZOOKEEPER_TLS_VERIFY_HOSTNAME=false \
-e KAFKA_ZOOKEEPER_TLS_TYPE=JKS \
-e KAFKA_TLS_TYPE=JKS \
-e KAFKA_CFG_ZOOKEEPER_CONNECT="zookeeper-host:2281" \
-e KAFKA_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.rmi.port=9999 -Djava.rmi.server.hostname=Private_IP -Djava.net.preferIPv4Stack=true" \
--name=kafka \
bitnami/kafka:3.2.1

Still, I got same error -

Caused by: org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/
        at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:512)
        at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:369)
        ... 23 more
Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/
        at org.apache.zookeeper.common.KeyStoreFileType.fromFilename(KeyStoreFileType.java:90)
        at org.apache.zookeeper.common.KeyStoreFileType.fromPropertyValueOrFileName(KeyStoreFileType.java:112)
        at org.apache.zookeeper.common.X509Util.loadTrustStore(X509Util.java:407)
        at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:488)
        ... 24 more
Mauraza commented 2 years ago

Hi @patiltussh,

Could you check the permission of the files? and add -e BITNAMI_DEBUG=true to check if appears more logs.

patiltussh commented 2 years ago

Hi @Mauraza

Files are having 777 permission as below -

pwd
/root/kafka/config/certs

:~/kafka/config/certs# ls -ltra
total 0
drwxrwxrwx 2 root root   6 Aug 29 13:00 zookeeper.truststore.jks
drwxrwxrwx 2 root root   6 Aug 29 13:00 zookeeper.keystore.jks
drwxrwxrwx 2 root root   6 Aug 29 13:00 kafka.truststore.jks
drwxrwxrwx 2 root root   6 Aug 29 13:00 kafka.keystore.jks
drwxr-xr-x 3 root root  19 Aug 29 13:00 ..
drwxr-xr-x 6 root root 122 Aug 29 13:00 .

I have also added -e BITNAMI_DEBUG=true flag. But it is generating below error. No more logs are appering -

Caused by: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/
        at org.apache.zookeeper.common.KeyStoreFileType.fromFilename(KeyStoreFileType.java:90)
        at org.apache.zookeeper.common.KeyStoreFileType.fromPropertyValueOrFileName(KeyStoreFileType.java:112)
        at org.apache.zookeeper.common.X509Util.loadTrustStore(X509Util.java:407)
        at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:488)
        ... 24 more
Mauraza commented 2 years ago

Hi @patiltussh,

Could you tell me this environment variable -e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,SSL://:9093 \?

The Bitnami Apache Kafka docker image disables the PLAINTEXT listener for security reasons. You can enable the PLAINTEXT listener by adding the next environment variable, but remember that this configuration is not recommended for production. 

 ALLOW_PLAINTEXT_LISTENER=yes

If you are using PLAINTEXT you need to enable it.

patiltussh commented 2 years ago

Hi @Mauraza KAFKA_CFG_LISTENERS environment variable value is defined as below - -e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,SSL://:9093

And ALLOW_PLAINTEXT_LISTENER environment variable value is defined as below - -e ALLOW_PLAINTEXT_LISTENER="yes"

Mauraza commented 2 years ago

Hi @patiltussh,

sorry, in the command you pasted did appear that the environment variable, could you try without "? -e ALLOW_PLAINTEXT_LISTENER=yes

patiltussh commented 2 years ago

Hi @Mauraza I tried without " as well. And I got same error. -e ALLOW_PLAINTEXT_LISTENER=yes

Complete command -

/usr/bin/docker run --rm --net=host \
--ulimit nofile="65536" \
-v /root/kafka/config/certs/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro \
-v /root/kafka/config/certs/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks:ro \
-v /root/kafka/config/certs/zookeeper.keystore.jks:/opt/bitnami/kafka/config/certs/zookeeper.keystore.jks:ro \
-v /root/kafka/config/certs/zookeeper.truststore.jks:/opt/bitnami/kafka/config/certs/zookeeper.truststore.jks:ro \
-p 9092:9092 \
-p 9093:9093 \
-p 9999:9999 \
-p 8083:8083 \
-p 8888:8888 \
-p 8889:8889 \
-e KAFKA_ZOOKEEPER_PROTOCOL=SSL \
-e KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,SSL://:9093 \
-e KAFKA_ZOOKEEPER_TLS_KEYSTORE_PASSWORD=test12 \
-e KAFKA_ZOOKEEPER_TLS_TRUSTSTORE_PASSWORD=test12 \
-e KAFKA_ZOOKEEPER_TLS_VERIFY_HOSTNAME=false \
-e KAFKA_ZOOKEEPER_TLS_TYPE=JKS \
-e KAFKA_TLS_TYPE=JKS \
-e ALLOW_PLAINTEXT_LISTENER=yes \
-e KAFKA_CFG_ZOOKEEPER_CONNECT="zookeeper_node:2281" \
-e KAFKA_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=9999 -Dcom.sun.management.jmxremote.rmi.port=9999 -Djava.rmi.server.hostname=Private_IP -Djava.net.preferIPv4Stack=true" \
--name=kafka \
bitnami/kafka:3.2.1

Error -

Caused by: org.apache.zookeeper.common.X509Exception$TrustManagerException: java.lang.IllegalArgumentException: Unable to auto-detect store file type from file name: /opt/bitnami/kafka/config/certs/
        at org.apache.zookeeper.common.X509Util.createTrustManager(X509Util.java:512)
        at org.apache.zookeeper.common.X509Util.createSSLContextAndOptionsFromConfig(X509Util.java:369)
Mauraza commented 2 years ago

Hi @patiltussh,

Could you try using this environment variable -e KAFKA_ZOOKEEPER_TLS_TRUSTSTORE_FILE=/opt/bitnami/kafka/config/certs/zookeeper.truststore.jks?

github-actions[bot] commented 1 year ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

patiltussh commented 1 year ago

Hi @Mauraza Adding below parameter helped and issue got fixed. Thank you. -e KAFKA_ZOOKEEPER_TLS_TRUSTSTORE_FILE=/opt/bitnami/kafka/config/certs/zookeeper.truststore.jks