cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.88k stars 3.77k forks source link

Easily differentiate a full backup from an incremental in the debug.zip and the DBConsole #95692

Open ccampbell-crl opened 1 year ago

ccampbell-crl commented 1 year ago

TSE has noticed difficulty in easily identifying if a backup in our crdb_internal.jobs.txt is a full backup or an incremental backup. Customers also may have difficulty identifying backup type by looking in the DBConsole.

The request is for engineering / Disaster Recovery to come up with a way to easily identify full backups vs incremental backups in both a debug.zip and the DBConsole.

One example would be to have two different “types” of backups in the crdb_internal.jobs.txt file in a debug, “Backup - Full” and “Backup - Incremental”

Jira issue: CRDB-23679

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/disaster-recovery

benbardin commented 1 year ago

This is already possible, though not obvious. A full backup has syntax BACKUP <target> INTO <storage> An inc backup has syntax BACKUP <target> INTO <full_backup> IN <storage> full_backup can either be LATEST, or an explicit directory in the storage bucket that contains a full backup. The key differentiator here is IN.

That said this is an extremely subtle distinguishing feature. We could consider ways to make this more observable.

dt commented 1 year ago

@benbardin Isn't the IN clause always in the description once the job has run?

When you look at the jobs after it has run, I thought I remembered that the description of the full is updated to reflect where it actually backed up to, i.e. it is shown as BACKUP INTO date-based-path IN collection, which is identical to if it had been an incremental to that location.

Also incremental will not show up as LATEST after they have been run: the job description is updated with the resolved path once it is resolved.

I believe there is a subtle way you can tell, which is that the AOST time will match the date-based-path time on the full, but not for subsequent later incremental into that same path, but that's even more subtle than the IN clause.

benbardin commented 1 year ago

When you look at the jobs after it has run, I thought I remembered that the description of the full is updated to reflect where it actually backed up to, i.e. it is shown as BACKUP INTO date-based-path IN collection, which is identical to if it had been an incremental to that location.

TIL!

I believe there is a subtle way you can tell, which is that the AOST time will match the date-based-path time on the full, but not for subsequent later incremental into that same path, but that's even more subtle than the IN clause.

🤣