search

vert-x3 / vertx-consul-client

Vert.x Consul Client
Apache License 2.0
34 stars 23 forks source link

Discovering services with DNS #34

Closed junglie85 closed 7 years ago

junglie85 commented 7 years ago

I have a question, which may be Vert.x related or it may be to do with my understanding of how Consul works. If the latter and you feel it not appropriate to discuss here (although it may benefit others), please suggest a better option.

As I understand it, service discovery can be done using DNS. That means that if I register the DNS address of Consul in the AddressResolverOptions of VertxOptions in beforeStartingVertx() in a custom Launcher, I should be able to set the address of whatever service I want to discover as something like kafka.service.consul (assuming I've registered Kafka with the agent) and it should return the correct IP address and port. However, using the vertx-kafka-client and setting the BOOTSTRAP_SERVERS option to kafka.service.consul I get an exception due to an invalid URL: Caused by: org.apache.kafka.common.config.ConfigException: Invalid url in bootstrap.servers: kafka.service.consul.

My question, then, first of all, is whether my understanding of how I should be able to use Consul correct? If not, then I guess that makes the rest of my questions mute. If it is, then: How do I check to make sure what AddressResolver's are registered with a given Vertx? I checked the vertx-core docs but couldn't see anything. Is this valid behaviour for the vertx-consul-client? Is this actually an issue with the vertx-kafka-client?

I ask this, because in my mind it should be possible to use the Consul DNS resolver to get the correct address of, in this instance, a Kafka instance. I'm currently doing a discovery and instantiating a new Kafka producer every time I want to send a message, which seems inefficient. Granted I could use the same service and register for changes, then create a new Kafka producer when needed. However, this still seems inefficient (although less so than my current implementation) and I want to check whether my understanding needs correcting or if it's a limitation with the vertx-consul-client (or possibly the vertx-kafka-client)?

Apologies for referring to the vertx-kafka-client so much, but it seemed to make sense to try and provide a concrete example of what I'm trying to achieve with Consul.

ruslansennov commented 7 years ago

My question, then, first of all, is whether my understanding of how I should be able to use Consul correct?

Both discovery modes (via DNS or via HTTP-requests) are valid and equals. But you need to use ConsulClient in the second case only, because this is a slim http proxy. If you want to use DNS discovery, you must properly configure your kafka client and this is not a vertx-consul-client issue

junglie85 commented 7 years ago

@ruslansennov Great, thanks very much for the clarification. I'll go and do some more digging into the kafka client and its configuration.