AbsaOSS / spline

Data Lineage Tracking And Visualization Solution
https://absaoss.github.io/spline/
Apache License 2.0
599 stars 154 forks source link

Automated transaction recovery #1155

Open wajda opened 1 year ago

wajda commented 1 year ago

In rare cases when a transaction was broken and didn't rollback properly (network partitioning between shards, sudden coordinator crash etc), another transaction has to be able to take up on where the previous one left off.

Possible solutions:

  1. Clean the previous transaction assets and start over.
  2. Overwrite the previous transaction data.

Challenges:

  1. How to distinguish between abandoned transaction and a long-running transaction in a distributed environment?
  2. How to kill stuck transaction?