ConsumerGroup unorganized data return

[aditi805]

node version : v8.0.0 Kafka version : 0.11.0.0 kafka-node : 2.0.1

Hi, I tried producing some data and consume the same from the mentioned versions and i am having some problem while consuming the data. I was using consumer group with the following options: options: { //host:'x.y.z.abc:2181', //kafkaHost : 'xxxxxxx:9092,yyyyyyyy:9092,zzzzzzzz:9092', groupId: 'h', autoCommit: false, fetchMaxWaitMs: 100, fetchMinBytes: 100, fetchMaxBytes: 1024*128, fromOffset: 'earliest', protocol: ['roundrobin'] },

Document in kafka : { "_id" : ObjectId("596c9e07baae5dda881b5edb"), "uuid" : 123, "data" : "yes, consuming" }

Consumed Document : 123,"data":"yes, consuming"}9e07baae5dda881b5edb","uu"� undefined:1 "M`@[ ^ SyntaxError: Unexpected token in JSON at position 0

• Number of Brokers: 3
• Number partitions for topic: 128

I tried the command line approach for consumer to check if the data is being produced properly. ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic singleDocTest --from-beginning {"_id":"596c9e07baae5dda881b5edb","uuid":123,"data":"yes, consuming"}

What could be the possible config/data error ?

consumerGroup.on('message', function (message) { console.log(message.value); //to check weird data on server randomArray.push(JSON.parse(message.value)); ... ... }

[aditi805]

hi @hyperlink . any update ?

[hyperlink]

Was compression enabled? There are some known issues dealing with compression.

[aditi805]

yes, lz4 compression on server side. is there a work around for this issue ?