search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.04k stars 3.79k forks source link

bulkio: protections against runaway incremental backup chains #69374

Closed mwang1026 closed 1 year ago

mwang1026 commented 3 years ago

You pause the schedule for your full backup. Your full backups keep failing because of a bug. You decide it's funny to cancel a full backup one day.

It's possible your incremental backup schedule gets longer and longer. We know that as those incremental chains get longer, bad things happen.

We should have a configurable setting that is a multiplier -- if the last full backup is X times older than the full backup frequency, start failing incremental schedules and signal that failure in clear observability for the user. The problem today is people look at jobs for their failures, but in this case we'd fail before creating a job. We should think through that UX

Jira issue: CRDB-9566

github-actions[bot] commented 1 year ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!