Neo4j Streams is not sending the data to kafka when the kafka server is restarting

[vishnu2497]

Hello Guys,

The following is my neo4j.conf

streams.source.enabled=true
streams.sink.enabled=false

kafka.zookeeper.connect=localhost:2181
kafka.bootstrap.servers=localhost:9092
kafka.acks=1
kafka.num.partitions=1
kafka.retries=2
kafka.batch.size=16384
kafka.buffer.memory=33554432
kafka.reindex.batch.size=1000
kafka.session.timeout.ms=15000
kafka.connection.timeout.ms=10000
kafka.replication=1
kafka.linger.ms=1
kafka.client.id=testif23
kafka.topic.discovery.polling.interval=5000

streams.source.topic.nodes.neo4jfailtest=neo4jfailtest{*}
streams.source.topic.nodes.neo4jfailtest1=neo4jfailtest1{*}
apoc.import.file.enabled=true
apoc.trigger.enabled=true
streams.procedures.enabled=true
dbms.security.procedures.whitelist=streams.*,apoc.*

it is working perfectly when everything works fine

But for checking purpose i turn off the kafka server and insert some data into the particular configured node in neo4j

when i am restarting the server and checked the kafka topic whatever data i inserted between the kafka down and restart is not inserted into the kafka topic

After 30 mins also data is not came into topic.

so is this the default behaviour?

Expected behaviour would be: Has to send the data whichever is not inserted as soon as kafka came up.

is there any solution for this?

[vishnu2497]

From stack overflow i got to know kafka.retries=2 so it is trying to retry only twice.

so how can i configure that it will retry continuously forever till the connection active.

i updated this config like below

kafka.acks=1
kafka.num.partitions=1
kafka.retries=99999
kafka.batch.size=16384
kafka.buffer.memory=33554432
kafka.reindex.batch.size=1000
#kafka.session.timeout.ms=15000
#kafka.connection.timeout.ms=10000
kafka.replication=1
kafka.linger.ms=1
kafka.client.id=testif23
kafka.topic.discovery.polling.interval=5000
kafka.retry.backoff.ms=5000
kafka.request.timeout.ms=5000
kafka.delivery.timeout.ms=99999999

[ali-ince]

Hi @vishnu2497, is this still an issue, even with the latest version of the plugin?