Host gets hashed in metadata?

[pulzzedavid]

I am trying to bring up a Kafka connection but is failing. Noticed that metadata has host as what seems to be a hashed value. Any pointers would be much appreciated. I am testing with a single Kafka instance from https://hub.docker.com/r/bitnami/kafka/.

iex(8)> KafkaEx.create_worker(:e, uris: [{"localhost", 9092}])
[debug] Successfully connected to broker "localhost":9092
[debug] Establishing connection to broker 1001: "2bb92465aad1" on port 9092
[error] Could not connect to broker "2bb92465aad1":9092 because of error {:error, :nxdomain}
{:ok, #PID<0.2302.0>}
iex(9)> KafkaEx.metadata(worker_name: :e) |> Map.get(:brokers)
[
  %KafkaEx.Protocol.Metadata.Broker{
    host: "2bb92465aad1",
    is_controller: nil,
    node_id: 1001,
    port: 9092,
    socket: nil
  }
]
iex(10)> KafkaEx.produce(
...(10)>   %KafkaEx.Protocol.Produce.Request{
...(10)>     topic: "test",
...(10)>     partition: 0,
...(10)>     required_acks: 1,
...(10)>     messages: [%KafkaEx.Protocol.Produce.Message{value: "Test message"}]
...(10)>   }, worker_name: :e
...(10)> )
[debug] Closing connection to broker 1001: "2bb92465aad1" on port 9092
[debug] Establishing connection to broker 1001: "2bb92465aad1" on port 9092
[error] Could not connect to broker "2bb92465aad1":9092 because of error {:error, :nxdomain}
:leader_not_available
[error] kafka_server_produce_send_request: leader for topic test/0 is not available
iex(25)>

[dantswain]

Hi @pulzzedavid !

When you make the initial "bootstrap" connection (your "localhost"), Kafka responds with its own list of brokers. This is what you see in kafka documentation referenced as the "advertised hostnames". The idea is that you can connect to a large cluster if you know only one of the hostnames (the host names can be dynamic in general, since nodes in the cluster can go online/offline).

It looks like you're getting the docker internal hostname. I'm not familiar with the image you're using, but it probably has a way to override the advertised host names, which is what you want here.

So to be clear - KafkaEx is not hashing the hostname. The value you're seeing is likely the docker container's internal hostname.

[pulzzedavid]

Thanks dantswain,

I see. In that case, let me close this issue and try to override the advertised host name. Much appreciate the pointer.