We plan to run new ingesters with delay between prepare-downscale and actual downscale. During this delay, ingesters will keep running, but will not receive any new data.
Timestamp of prepare-downscale call is recorded as Pod annotation.
After delay elapses (checked as max recorded timestamp in pod annotations), rollout operator's admission controller allows final downscale.
User can cancel the downscale during this delay, and scale up again. In that case rollout operator can be configured to call DELETE on prepare-downscale endpoint.
We plan to run new ingesters with delay between prepare-downscale and actual downscale. During this delay, ingesters will keep running, but will not receive any new data.
Timestamp of prepare-downscale call is recorded as Pod annotation.
After delay elapses (checked as max recorded timestamp in pod annotations), rollout operator's admission controller allows final downscale.
User can cancel the downscale during this delay, and scale up again. In that case rollout operator can be configured to call
DELETE
on prepare-downscale endpoint.(draft, doing manual testing)