search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.99k stars 1.07k forks source link

[DocDB] Backups of YSQL DBs taken when an index is invalid are corrupt #15184

Open druzac opened 1 year ago

druzac commented 1 year ago

Jira Link: DB-4373

Description

Backups taken of a YSQL db with an index marked invalid in the pg_index table can be corrupt. The issue is the sql dump produced does not include the index. However the SnapshotInfoPB produced by the export_snapshot command does include metadata for the index. This results in a failed restore when using the backup and an error message like:

W1127 05:35:03.689646 21917 catalog_manager_ent.cc:2233] ImportTableEntry: YSQL table not found: customer
druzac commented 1 year ago

We could fix this at restore time by ignoring indices present in SnapshotInfoPB that don't exist when handling a import_snapshot request. We could try to fix this at backup time by ignoring indices that are invalid in export_snapshot but that introduces issues if the validity of an index changes between ysqldump time and snapshot export time when doing a backup.

If we ignore indices we should ideally notify users somehow.