ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$) java.lang.NullPointerException

[funky-eyes]

What can we help you with?

After referring to the yaml file of minio, I tried to configure the relevant configuration items for tiered storage, but it throws a npe exception on startup.

log.retention.minutes=10
log.local.retention.ms=300000
log.segment.bytes=104857600
log.retention.check.interval.ms=300000
remote.log.metadata.topic.replication.factor=1
remote.log.storage.system.enable=true
remote.log.metadata.topic.retention.ms=-1
rsm.config.fetch.chunk.cache.class=io.aiven.kafka.tieredstorage.fetch.cache.DiskChunkCache
rsm.config.fetch.chunk.cache.path=/data01/kafka-tiered-storage-cache
rsm.config.storage.backend.class=io.aiven.kafka.tieredstorage.storage.s3.S3Storage
rsm.config.storage.s3.bucket.name=kafka
rsm.config.storage.s3.region=kafka
rsm.config.storage.s3.endpoint.url=http://xxxx:9000
rsm.config.storage.aws.secret.access.key=xxxxx
rsm.config.storage.aws.access.key.id=kafka
remote.log.metadata.manager.class.path=/home/admin/core-0.0.1-SNAPSHOT/*:/home/admin/s3-0.0.1-SNAPSHOT/*
remote.log.metadata.manager.class.name=io.aiven.kafka.tieredstorage.RemoteStorageManager

[2024-03-04 16:32:40,908] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
java.lang.NullPointerException
    at java.base/java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
    at java.base/java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1541)
    at java.base/java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:667)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:651)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    at kafka.log.remote.RemoteLogManager.createDelegate(RemoteLogManager.java:225)
    at kafka.log.remote.RemoteLogManager.access$200(RemoteLogManager.java:132)
    at kafka.log.remote.RemoteLogManager$2.run(RemoteLogManager.java:243)
    at kafka.log.remote.RemoteLogManager$2.run(RemoteLogManager.java:234)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
    at kafka.log.remote.RemoteLogManager.createRemoteStorageManager(RemoteLogManager.java:234)
    at kafka.log.remote.RemoteLogManager.<init>(RemoteLogManager.java:194)
    at kafka.server.KafkaServer.createRemoteLogManager(KafkaServer.scala:676)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:320)
    at kafka.Kafka$.main(Kafka.scala:112)
    at kafka.Kafka.main(Kafka.scala)

Where would you expect to find this information?

[jeqo]

Seems like you're using the wrong configs:

remote.log.metadata.manager.class.path=/home/admin/core-0.0.1-SNAPSHOT/*:/home/admin/s3-0.0.1-SNAPSHOT/*
remote.log.metadata.manager.class.name=io.aiven.kafka.tieredstorage.RemoteStorageManager

they should be:

remote.log.storage.manager.class.path=/home/admin/core-0.0.1-SNAPSHOT/*:/home/admin/s3-0.0.1-SNAPSHOT/*
remote.log.storage.manager.class.name=io.aiven.kafka.tieredstorage.RemoteStorageManager

Have a look into the demos and have a look into the configurations used to deploy.

[funky-eyes]

Seems like you're using the wrong configs:

remote.log.metadata.manager.class.path=/home/admin/core-0.0.1-SNAPSHOT/*:/home/admin/s3-0.0.1-SNAPSHOT/*
remote.log.metadata.manager.class.name=io.aiven.kafka.tieredstorage.RemoteStorageManager

they should be:

remote.log.storage.manager.class.path=/home/admin/core-0.0.1-SNAPSHOT/*:/home/admin/s3-0.0.1-SNAPSHOT/*
remote.log.storage.manager.class.name=io.aiven.kafka.tieredstorage.RemoteStorageManager

Have a look into the demos and have a look into the configurations used to deploy.

I'm sorry, I was dizzy and didn't see the names of the configuration items clearly. Currently, there are indeed quite a few configuration items that need attention.