idempotence setting does not seem to work

[guyappy]

Environment Information

• OS [e.g. Mac, Arch, Windows 10]: Windows 10
• Node Version [e.g. 8.2.1]: 18.17.1
• NPM Version [e.g. 5.4.2]: 9.6.7
• C++ Toolchain [e.g. Visual Studio, llvm, g++]: VS
• node-rdkafka version [e.g. 2.3.3]: 2.9.1

Steps to Reproduce I configured a producer as follows:

const producer = new Kafka.Producer({ ...ClientConfig, 'dr_msg_cb': true, retries: 10, ['message.max.bytes'] : 1000000, // 1mb ['enable.auto.commit'] : true, 'enable.idempotence' : true, 'acks' : 'all', // this value is automatically overwritten by 'enable.idempotence' 'max.in.flight.requests.per.connection' : 5, 'queuing.strategy' : 'fifo', 'security.protocol': 'ssl', 'ssl.certificate.location': ".......cer", 'ssl.keystore.location': ".....pfx", 'ssl.keystore.password': process.env.ESP_KEYSTORE_PASSWORD_DT, 'ssl.ca.location': '.....pem',
'client.id': 'redacted', });

I then produce my messages and trying to produce them in order: for (var i = 0; i < maxMessages; i++) { var value = Buffer.from(key-customer${randomIntFromInterval(1,4)}-${i}); var key = key-customer${randomIntFromInterval(1,4)}; // if partition is set to -1, librdkafka will use the default partitioner var partition = -1; var headers = [ { header: "header value" } ] console.log(produced message with key ${key} and message ${value}); producer.produce(topicName, partition, value, key, Date.now(), "", headers); await sleep(100); }

this produces the following messages in order:

when I later consume these messages, using

stream.on('data', (message) => { console.log('Got message: ' + message);

they are not consumed in the order that I produced them;

I'm really not sure what I'm doing wrong. Any help is greatly appreciated.

[guyappy]

sorry my bad, I didnt set the right key value for each of my customers in this example. after setting the same key per customer this worked fine.