Duplicate records Inserted

[dbhatia2]

Here Setup Maxwell is reading from mysql binary logs and sending it to kafka.

• From kafka, we are using this kafka-connect-mongodb, version 9 to import to Mongodb
• listening at kafka consumer, we only see 1-1 message.
• In Mongo for every message is produced, we are seeing 2 entries pushed.

Can you please suggest, Let me know if you need more logs.

more mongo_connector_configs.json.bkp { "name":"mongo-connector-", "config" :{ "connector.class":"com.startapp.data.MongoSinkConnector", "tasks.max":"5", "db.host":"*", "db.port":"27017", "db.name":"**", "db.collections":"mysqlcdc", "write.batch.enabled":"true", "write.batch.size":"200", "connect.use_schema":"false", "topics":"maxwell" } }

more /tmp/connect-distributed.properties|grep -v '#'

bootstrap.servers=localhost:9092

group.id=connect-cluster

key.converter=org.apache.kafka.connect.json.JsonConverter value.converter=org.apache.kafka.connect.json.JsonConverter key.converter.schemas.enable=false value.converter.schemas.enable=false

internal.key.converter=org.apache.kafka.connect.json.JsonConverter internal.value.converter=org.apache.kafka.connect.json.JsonConverter internal.key.converter.schemas.enable=false internal.value.converter.schemas.enable=false

offset.storage.topic=mongo-connect-offsets offset.flush.interval.ms=10000 config.storage.topic=mongo-connect-configs

bin/kafka-topics.sh --list --zookeeper localhost:2181 __consumer_offsets maxwell mongo-connect-configs mongo-connect-offs - marked for deletion mongo-connect-offsets test - marked for deletion

Thanks Dk

[itayhasson12]

Hi! Have you solved this bug?