flightstats / hub

fault tolerant, highly available service for data storage and distribution
http://www.flightstats.com
MIT License
103 stars 35 forks source link

Update webhook LAST_COMPLETED ZK node with the minute before the first failed s3 verifier update #1132

Closed lkemmerer closed 5 years ago

lkemmerer commented 5 years ago

I resisted the urge to streamify this logic, so (yay) I introduced more mutable state!

This will assume we update to the end range of the verifier run, but if it encounters an error, it'll set last completed to the minute before the failure. If that means that it got past the first minute, we'll update the ZK node.

~Question: should we update it if the failure happens in the first minute after the start path (i.e we start the range at 12:00 and have a failure somewhere in the items at 12:01, should we update last completed to 12:00?)~ (LAST_COMPLETED minute is exclusive for spoke, so no need to do this)

crozierm commented 5 years ago

Look legit to me.