(This includes the changes from #171, #173, #176, #175 and builds on top. These should likely be merged first, then this one gets smaller. Implements parts of #170.)
This introduces an optional "delete-after-insert" mode. In this mode, events stored into the database will be immediately deleted.
The purpose of this is to connect the eventlog table to Fabric Event Stream's Debezium-connector (this is a Zalando-internal project), which will read the events from the logical replication log and takes care of sending them out to Nakadi.
Relevant changes:
171: Introduce plural method for deletion into EventLogRepository, and use it from the EventTransmissionService. (Also refactors the persist() part a bit.) (by now already in master)
176
175
173
The new EventLogRepositoryImpl.persistWithIds() methods will now update the id field in the EventLog objects (implemented using the QueryStatementBatcher from ↑).
New method EventLogRepository.persistAndDelete(), which, as the name implies, calls persist() and then delete(). In EventLogRepositoryImpl, it instead calls persistWithIds, so it actually works.
New option nakadi-producer.delete-after-write for the spring configuration which, when true, will make EventLogWriter use persistAndDelete() instead of persist().
Some javadoc in EventLogRepository
Explanation in the README.
179
create the publication as part of the migration scripts.
TODO:
[ ] actually try this out – it seems to work locally in an example project I made. I was not able to deploy this to a cluster yet, as the corresponding change to FES is not yet deployed.
(This includes the changes from #171, #173, #176, #175 and builds on top. These should likely be merged first, then this one gets smaller. Implements parts of #170.)
This introduces an optional "delete-after-insert" mode. In this mode, events stored into the database will be immediately deleted.
The purpose of this is to connect the eventlog table to Fabric Event Stream's Debezium-connector (this is a Zalando-internal project), which will read the events from the logical replication log and takes care of sending them out to Nakadi.
Relevant changes:
171: Introduce plural method for deletion into EventLogRepository, and use it from the EventTransmissionService. (Also refactors the persist() part a bit.) (by now already in master)
176
175
173
EventLogRepositoryImpl.persistWithIds()
methods will now update theid
field in the EventLog objects (implemented using the QueryStatementBatcher from ↑).EventLogRepository.persistAndDelete()
, which, as the name implies, calls persist() and then delete(). In EventLogRepositoryImpl, it instead calls persistWithIds, so it actually works.nakadi-producer.delete-after-write
for the spring configuration which, when true, will make EventLogWriter usepersistAndDelete()
instead ofpersist()
.179
TODO: