mesos / kafka

Apache Kafka on Apache Mesos
Apache License 2.0
414 stars 140 forks source link

More config support for scheduler from Apache Kafka? #308

Closed akitekt closed 7 years ago

akitekt commented 7 years ago

It seems the scheduler does not support enough configs like the apache kafka. Does it mean these options are not supported?

For example, in apache kafka server. properties, you can set :   | auto.create.topics.enable=false (Critical for Production usage)

More options : https://github.com/apache/kafka/blob/trunk/config/server.properties

steveniemitz commented 7 years ago

You can set all kafka server properties via --options

akitekt commented 7 years ago

Hi @steveniemitz the scheduler complains with the error below when using --options: Exception in thread "main" java.lang.Error: options is not a recognized option Here is how i used it: /kafka-mesos.sh scheduler --master=xxx --zk=xxx --api=xxx --storage=zk:/kafka-mesos-scheduler --framework-name=kafka --framework-timeout=30d --log=/var/log/kafka-mesos.log --debug=false --options auto.create.topics.enable=false

akitekt commented 7 years ago

@steveniemitz for version, Kafka Jar: kafka-mesos-0.10.0.0-rc2-kafka_2.10-0.10.1.1.jar

steveniemitz commented 7 years ago

I recommend reading the documentation: https://github.com/mesos/kafka#adding-brokers-to-the-cluster

It's a broker configuration option, you're trying to set it on the scheduler itself.

veeruns commented 7 years ago

kafka-mesos.sh broker add 0 --options file:../server.properties

This is what I did

akitekt commented 7 years ago

Thanks guys, i was thinking if there is a consistent way to force all brokers to follow the same options. I think i will use @veeruns solution with versioned properties files for now.