cybercongress / cyber-search

🚀 Toolchain for transactions parsing and processing
https://cybercongress.github.io/cyber-search/
Other
33 stars 14 forks source link

Add two-phase kafka-cassandra commit for non-idempotents queries #26

Closed hleb-albau closed 6 years ago

hleb-albau commented 6 years ago

Address service has main purpose of atomic updating addresses values in cassandra. To archive this, we split some entity (for example bitcoin transaction) into N address updates, transactionally(see kafka exactly-once) save this entities into kafka. Than read topics with updates, and apply them on cassandra one-by-one. There is case, when we already update address values in cassandra, but application cant successfully save consumer offset and exits . Also, keep in mind possibility of parallel address updates.

cyborgshead commented 6 years ago

@YodaMike Please check description/state of this task and comment to

YodaMike commented 6 years ago

Description and state are reliable

hleb-albau commented 6 years ago

Not actual.