search

elodina / dropwizard-kafka-http

Apache Kafka HTTP Endpoint for producing and consuming messages from topics
http://www.elodina.net
Apache License 2.0
153 stars 45 forks source link

how to create multiple broker ? #5

Closed ghost closed 10 years ago

ghost commented 10 years ago

As we all know, we use apache kafka usually because of the fault tolerance, but one broker is not going to do that. Is this project implemented creating multiple broker ?

joestein commented 10 years ago

You can pass as many brokers you want to metadataBrokerList host:port,host:port:host:port. The way Kafka works is the broker list is used for service discovery not for actually producing. So if you have 20 brokers you can still just pass 2 brokers to the producer list and be good to go. The KafkaConfiguration simply passes this property https://github.com/stealthly/dropwizard-kafka-http/blob/master/src/ly/stealth/kafkahttp/KafkaConfiguration.java#L20 through. You can look at the Kafka documentation for specific details https://kafka.apache.org/documentation.html#producerconfigs "This is for bootstrapping and the producer will only use it for getting metadata (topics, partitions and replicas). The socket connections for sending the actual data will be established based on the broker information returned in the metadata. The format is host1:port1,host2:port2, and the list can be a subset of brokers or a VIP pointing to a subset of brokers."