Publisher topic generates successfully but message doesn't post.

[dsalazar-carecloud]

Consumer works like a charm but the producer is another story. When attempting to post to a topic the library will generate the topic, if not already there, but the message to be posted never posts. Infact I get the following error.

/Users/foo/.rbenv/versions/2.1.0/gemsets/kafka/gems/poseidon-0.0.5/lib/poseidon/sync_producer.rb:66:in `send_messages': Failed to send all messages: [#<Poseidon::Message:0x007fbb7a9f1ef0 @struct=#<struct Poseidon::Protocol::MessageWithOffsetStruct offset=0, message=#<struct Poseidon::Protocol::MessageStruct magic_type=0, attributes=0, key=nil, value="hello">>, @topic="test">] remaining (RuntimeError)
    from /Users/foo/.rbenv/versions/2.1.0/gemsets/kafka/gems/poseidon-0.0.5/lib/poseidon/producer.rb:163:in `send_messages'
    from producer.rb:6:in `<main>'

My implementation is very simple.

require 'poseidon'
producer = Poseidon::Producer.new(['foo:9092'], 'test_producer')
message_buffer = []
message_buffer << Poseidon::MessageToSend.new('test_topic', 'bar')
producer.send_messages(message_buffer)

Kafka install: kafka_2.9.2-0.8.1.1

Please advise...

[destagia]

I have the same problem...

[destagia]

I tried to send message with other languages, Scala, Node.js and Java. but all of them failed to send message to Kafka on external server.

So I think our problem occurs due to server settings.

[divyesh2506]

I too have the same problem :P . Please let me know if you find the cause or the solution.

[destagia]

I solved this. and found poseidon has no problems... ^^; If you use Kafka with docker-kafka, I can tell you the solution.

[divyesh2506]

Hey @tagia0212 ,the issue resolved by itself ,once I restarted the kafka server. But, I couldn't find the reason behind the error. Though I do not use docker, but some insight on the issue might help.

[daluu]

@tagia0212, could you elaborate on the solution for docker-kafka? Just curious to know.

[destagia]

@daluu OK, but I know the solution for kafka-docker, not docker-kafka. sorry.

I cloned the repository onto the server, and run docker up -d. When I failed to send topic from Poseidon to Kafka, I have not set config. So, I rewrite docker-compose.yml. environment, KAFKA_ADVERTISED_HOST_NAME must be set global IP Address of the server!

After I rerun the docker container, I could send the topics to Kafka.

[rhardih]

Not a fix, but at least it would then be possible to capture an explicit error instead of a RuntimeError: https://github.com/bpot/poseidon/pull/101

[mensfeld]

this would be useful to merge