zalando-nakadi / nakadi-ui

Official Nakadi user interface.
MIT License
69 stars 12 forks source link

Point-In-Time Subscription Reset Capability #39

Open soceanainn opened 5 years ago

soceanainn commented 5 years ago

Suggesting pinned offsets at scheduled times (once daily, or part thereof - tag the latest offset for each partition).

This will make it very easy for subscribers to recover after incidents, without having to search for the offsets. It will be used for "point-in-time" subscription reset.

It's a small scheduled job with very little overhead and would greatly reduce a lot of the support we have to provide our consumers for finding the appropriate offsets and for minimising the recovery time after incidents.

soceanainn commented 5 years ago

A nice solution could be to provide hourly pinned offsets for the previous 24 hours, as well as daily pinned offsets for the previous 7 days