search

neo4j-contrib / neo4j-streams

Neo4j Kafka Connector
https://neo4j.com/docs/kafka
Apache License 2.0
173 stars 71 forks source link

streams producer depreciation #517

Closed bwinchester closed 2 years ago

bwinchester commented 2 years ago

Reading the 4.1 release docs about the streams.conf and streams producer depreciation. Wanted to understand what drove that decision. Lack of use/feedback? underlying Neo4j 4.4 Java API changes or Neo4j 5.0 feature changes?

The push to kafka connect producer, which uses polling and timestamps on everything seems error prone and very difficult to diff changes. Whereas the CUD format on the transaction API ( Neo4j lists as one of four future proof classes) seems much less noisy on network traffic and fits the stream methodology much better to actual changes on your data triggering eventing.

We're thinking of forking the streams in-database jar. From our read, it encapsulates the streams.conf, reloading conf, get/set/remove lifecycles, and the CUD production, and works with 4.4.2.

danielnitsche commented 2 years ago

Any updates on this? And any further thoughts about forking @bwinchester?

bwinchester commented 2 years ago

I spoke with my enterprise Neo4j representatives, and they backed off the depreciation, saying they would support this connector until 5.x release in 2022. Neo4j 5.x is supposed to have (eventually) a replacement for streams or CUD eventing built into the db again, or in some way supported, and not using this db plugin. I'm just a user though and don't work for Neo4j. They might have changed opinions since I talked to them Early in 2022.