WallarooLabs / wally

Distributed Stream Processing
https://www.wallaroolabs.com
Apache License 2.0
1.48k stars 69 forks source link

Fix race between 1st checkpoint token, 2PC intro done, barrier complete #3074

Closed slfritchie closed 4 years ago

slfritchie commented 4 years ago

See comment added to ConnectorSink2PC.barrier_complete() for the details of the race between getting the 1st checkpoint token, reconnecting to the external connector sink, and getting the last checkpoint token and then calling barrier_complete().

When we hit this rare race, then we send a Phase1 message if needed and then (when the Phase1 reply arrives), force a Phase2 abort.

Fixes #3073