search

scylladb / scylla-manager

The Scylla Manager
https://manager.docs.scylladb.com/stable/
Other
48 stars 33 forks source link

Restore schema with internals #3876

Closed Michal-Leszczynski closed 1 week ago

Michal-Leszczynski commented 3 weeks ago

This PR makes backup/restore procedure use DESCRIBE SCHEMA WITH INTERNALS.

Backup: The schema.tar.gz (describe schema from driver method) is no longer backed up, as it was never used anyway. Instead of that, new schema_with_internals.json.gz (describe schema from cql query) is backed up and (when present) used for restore. This requires cluster credentials, so schema is backed up only when the credentials are set. If credentials are not there, backup proceed without DESCRIBE SCHEMA output. As a safety measure, system_schema is still always backed up so that it could be used to restore schema when DESCRIBE SCHEMA output is missing from the backup (this would require restoring system_schema into a single node cluster and then describing schema on it).

Restore schema: It always tries to use schema_with_internals.json.gz if present. Otherwise it checks Scylla in order to validate restore schema from sstables safety.

Fixes #3868 Fixes #3662

Michal-Leszczynski commented 2 weeks ago

@karol-kokoszka I addressed the comments (either by new commits or responses with questions).

zimnx commented 2 weeks ago

We (Operator team) would appreciate prioritizing this PR and preparing a release right after it's merged. This blocks our adoption of ScyllaDB 6.0.0, as Manager rejects restore attempt when 6.0.0 is used. Lack of support for it breaks our e2e tests.

karol-kokoszka commented 2 weeks ago

@Michal-Leszczynski Sanity-check is failing (unit-test).

Please check/fix.