Hangs when creating topic with Kafka 2.0.0 inside docker

[srfrnk]

Complete source code and environment is here When running with Kakfa 1.1.0 all works. Changing to Kafka 2.0.0 breaks the code. It hangs while trying to create the topic. Moreover - changing back to 1.1.0 after that will not fix things and you must restart-docker so that code starts to run.

[hyperlink]

Likely because BaseProducer is still using the old parameter with "async" boolean.

Workaround is to call the KafkaClient directly to create the topic.

[srfrnk]

Thanks for the info @hyperlink Sadly it doesn't help. I updated the repo with your suggestion. You can make sure it's what you meant. Any more ideas?

[hyperlink]

Did you push your changes?

Using the KafkaClient should work these tests are passing.

[srfrnk]

Pushed the changes just now. Sorry. You can see the code now.

On Tue, Sep 11, 2018 at 5:29 PM Xiaoxin Lu notifications@github.com wrote:

Did you push your changes?

Using the KafkaClient should work these tests https://travis-ci.org/SOHU-Co/kafka-node/jobs/426821436#L1631 are passing.

— You are receiving this because you authored the thread.

Reply to this email directly, view it on GitHub https://github.com/SOHU-Co/kafka-node/issues/1073#issuecomment-420294338, or mute the thread https://github.com/notifications/unsubscribe-auth/ABbp1k6xwN4FMzsS1hiECCqP_T5LtXuxks5uZ8i6gaJpZM4WgORf .

[hyperlink]

@srfrnk you need to pass an array of objects with the following fields topic, partitions and replicationFactor see the readme.

[srfrnk]

Updated to send

@srfrnk you need to pass an array of objects with the following fields topic, partitions and replicationFactor see the readme.

Done... + added client.connect just in case.... Now throws:

TypeError: Cannot read property 'host' of undefined
    at KafkaClient.getController (/app/node_modules/kafka-node/lib/kafkaClient.js:257:44)
    at KafkaClient.sendControllerRequest (/app/node_modules/kafka-node/lib/kafkaClient.js:1102:8)
    at KafkaClient.createTopics (/app/node_modules/kafka-node/lib/kafkaClient.js:903:8)
    at Promise (/app/kafka-node-test.js:16:16)
    at new Promise (<anonymous>)
    at kafkaTest (/app/kafka-node-test.js:13:11)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:182:7)

[rafkhan]

Has this been resolved, I'm running into the same issue.

[pavlovdog]

I'm running into the same issue:

const uuid = require('uuid');

const kafka = require("kafka-node");
const client = new kafka.KafkaClient();

const topic1 = uuid.v4();
const topic1ReplicationFactor = 1;
const topic1Partitions = 5;
const topic2 = uuid.v4();
const topic2ReplicationFactor = 1;
const topic2Partitions = 1;

client.connect();
client.createTopics(
  [
    {
      topic: topic1,
      partitions: topic1Partitions,
      replicationFactor: topic1ReplicationFactor
    },
    {
      topic: topic2,
      partitions: topic2Partitions,
      replicationFactor: topic2ReplicationFactor
    }
  ], (err, data) => console.log(err, data)
);

The error is:

/home/pavlovdog/.........../node_modules/kafka-node/lib/kafkaClient.js:282
    var broker = this.getBroker(controllerMetadata.host, controllerMetadata.port);
                                                   ^

TypeError: Cannot read property 'host' of undefined

Pretty strange, because I've taken this code directly from the test (link). I'm running Kafka with https://github.com/wurstmeister/kafka-docker, just docker-compose up -d, without any extra settings.