/data has been pre-populated with lost+found

[MazizEsa]

This happens when you run on AWS and attach a volume to the container from EBS. Kafka aren't able to start and spew the following errors:

2016-12-05T08:31:27.376226865Z [2016-12-05 08:31:27,376] ERROR There was an error in one of the threads during logs loading: kafka.common.KafkaException: Found directory /data/lost+found, 'lost+found' is not in the form of topic-partition 2016-12-05T08:31:27.376241733Z If a directory does not contain Kafka topic data it should not exist in Kafka's log directory (kafka.log.LogManager) 2016-12-05T08:31:27.380131877Z [2016-12-05 08:31:27,377] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 2016-12-05T08:31:27.380146851Z kafka.common.KafkaException: Found directory /data/lost+found, 'lost+found' is not in the form of topic-partition 2016-12-05T08:31:27.380152987Z If a directory does not contain Kafka topic data it should not exist in Kafka's log directory 2016-12-05T08:31:27.380157911Z at kafka.log.Log$.throwException(Log.scala:1112) 2016-12-05T08:31:27.380162402Z at kafka.log.Log$.parseTopicPartitionName(Log.scala:1100) 2016-12-05T08:31:27.380166440Z at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$10$$anonfun$apply$1.apply$mcV$sp(LogManager.scala:147) 2016-12-05T08:31:27.380170939Z at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:58) 2016-12-05T08:31:27.380174922Z at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 2016-12-05T08:31:27.380178913Z at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2016-12-05T08:31:27.380182792Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 2016-12-05T08:31:27.380186695Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 2016-12-05T08:31:27.380199767Z at java.lang.Thread.run(Thread.java:745) 2016-12-05T08:31:27.380675782Z [2016-12-05 08:31:27,380] INFO shutting down (kafka.server.KafkaServer)

[ches]

Hi,

lost+found is typically created by fsck at the root of filesystems, often on an unclean reboot. If that's the case with your EBS volume and Kafka is complaining about it, then you may simply want to use a child directory of the volume (e.g. /data/kafka) and mount that as the Docker containers' volume. So an argument like --volume /data/kafka:/data.

If I'm misunderstanding the problem let me know, but lost+found is coming from the Docker host OS, not something the container is creating or knows anything specifically about.

[DGBlock]

Hi,

In the stateful sets, you need to update the command at the time of container execution like below:

"rm -rf /opt/kafka/data/lost+found && exec /opt/kafka/bin/kafka-server-start.sh...."

This will remove the lost+found topics from the container and the kafka pods will spin up successfully.

This worked for me.