Closed nylqd closed 4 years ago
Hi @nylqd, could you please try with the jar provided here?
Hi @nylqd, could you please try with the jar provided here?
thx for ur reply, I have tried the official 4.0.3 release package, and still stuck at write data at target instance.
I have done to step to verify, 1) delete a node from source instance, see reading data from topic, query statistics: <nothing happened>
in target instance neo4j.log file; 2) create a node from source instance, see only reading data fro topic
and then stuck here forever.
Could you please share your neo4j.conf? In particular the streams configurations? Thanks in advance
here are the streams conf, hope it helps
# source instance conf in neo4j.conf
kafka.zookeeper.connect=....
kafka.bootstrap.servers=....
streams.sink.enabled=false
streams.source.enabled=true
streams.source.topic.nodes.<our topic>=<pattern>
streams.source.topic.relationships.<our topic>=<pattern>
# target instance conf in neo4j.conf
kafka.zookeeper.connect=....
kafka.bootstrap.servers=....
streams.source.enabled=false
streams.sink.enabled=false
streams.sink.enabled.to.<our db name>=true
streams.sink.topic.cdc.schema.to.<our db name>=<our topic>
streams.sink.errors.tolerance=all
streams.sink.errors.log.enable=true
streams.sink.errors.log.include.messages=true
@nylqd following the step you have provided i'm not able to reproduce the issue. I'm using the official 4.0.3 release package. However i noticed a difference between my source instance configuration and yours. Could you please add in the neo4j.conf of your source instance the following property and retry?
streams.source.schema.polling.interval=10000
@nylqd Any news? Thanks
@nylqd Any news? Thanks
@mroiter-larus sorry for the delay, have been working on other project. I have just apply the config u suggest, and the problem still exists.
I'm wondering this might be an issue with write data to neo4j? target instance log shows reading data from topic
then nothing but stuck.
I have test with another scenario all the same but only target instance with an empty db, and it works . The scenario described above is what we want, we just want sync data of an existing db to a different neo4j instance, is there any other solution may meet our need?
thx ur reply a lot.
@nylqd Sorry for the delay. Could you please try with the latest pre-release? You can find it here.
Thanks in advance!
@nylqd did you create the constraints on the target database?
@nylqd PING
@mroiter-larus @conker84 thx for ur reply
@nylqd Sorry for the delay. Could you please try with the latest pre-release? You can find it here.
Thanks in advance!
been busy on another project, I'd like to try out the pre-release in tomorrow's office hour.
@nylqd did you create the constraints on the target database?
what I have done is using neo4j-admin backup from the source db and restore to the target db, last time I checked, the constraints are intact.
Since we use it in our customers would be great dive into your issue to understand where is the problem. Please @nylqd let us know if you're still interested in this or we can close it
I discovered that when I config neo4j-streams sink to consume
cdc schema
event from kafka, the sink execution is not working properly.Expected Behavior (Mandatory)
all the
create/update/delete
cdc events should be handle properly.Actual Behavior (Mandatory)
create
event hang in thedb.execute
lambda, will not return thequeryStatistics
and proceed. the sink daemon job just hang in there and not consume any data from kafka.How to Reproduce the Problem
Steps (Mandatory)
neo4j-admin restore
to restore data from other db.cdc schema
strategy, config log level toDEBUG
write
of classStreamsEventSinkQueryExecution
to log the detail of query and params, no other change.4.0
.Screenshots (where it's possibile)
I have add debug log in write function and you may see the log above, the result of the execute lambda is showing, but the service just hang in there, will not proceed.
Specifications (Mandatory)
Currently used versions
Versions