Callback not called when message sending failed by High Level Producer

[haseebnaseem]

Questions?

• Check the Troubleshooting / FAQ ?
• Search the issues ?

Bug Report

Due to some problem with the cluster, which we are trying to locate, the zookeeperd are throwing the following error

ERROR Failed to send last message. Shutting down thread. (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.SocketException: Socket closed
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:118)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:134)
    at java.io.DataOutputStream.writeInt(DataOutputStream.java:197)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.send(QuorumCnxManager.java:889)
    at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:916)

Environment

• Node version: 7.10.1
• Kafka-node version:2.3.2
• Kafka version:1.0.0

For specific cases also provide

• Number of Brokers:3
• Number partitions for topic:3

Include Sample Code to reproduce behavior

// include code here
function updateFeedMessage(message){
    producer.send([{topic:'updateFeed',messages:JSON.stringify(message),partition:0}],function(err,data){
        if(err)
            console.log('error in sending update Feed message',err);
        else
            console.log('update feed message sent',data);
    });
}

Include output with Debug turned on

Thanks for your contribution!

[hyperlink]

Have you tried using KafkaClient instead of Client? KafkaClient does not use zookeeper.

[haseebnaseem]

I have tried using that but in that case the client does not detect even when the broker is up after a crash

[aikar]

@haseebnaseem the expectation would be that you would need to handle retrying the send again yourself.

[haseebnaseem]

I understand that if the sending failed, there should be a retrying mechanism however that would only happen if the callback is called with an error.

On Mon, 9 Apr 2018, 23:02 Daniel Ennis, notifications@github.com wrote:

@haseebnaseem https://github.com/haseebnaseem the expectation would be that you would need to handle retrying the send again yourself.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SOHU-Co/kafka-node/issues/908#issuecomment-379840819, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ1EQBptj9a-HbzZ-0jSfX7BMMHaDrpFks5tm6I6gaJpZM4S65jD .

[aikar]

@haseebnaseem that was in reference for if you use KafkaClient, you said you aren't using it since it doesn't detect when its up.

Im saying use KC, and keep retrying.

[haseebnaseem]

My kafka set up does not work without zookeeper, so once its down kafkaClient does not detect its up ever.

On Tue, 10 Apr 2018, 00:09 Daniel Ennis, notifications@github.com wrote:

@haseebnaseem https://github.com/haseebnaseem that was in reference for if you use KafkaClient, you said you aren't using it since it doesn't detect when its up.

Im saying use KC, and keep retrying.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SOHU-Co/kafka-node/issues/908#issuecomment-379860941, or mute the thread https://github.com/notifications/unsubscribe-auth/AJ1EQMcPm7BrJrEWgZWcZiPgyIvh_uXfks5tm7HRgaJpZM4S65jD .