Altinity / clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.
https://altinity.com
Other
1.25k stars 225 forks source link

`restore` command for ReplicatedMergeTree with --restore-database-mapping, should check replication path is exists in ZK and generate new path with predefined format, i.e. /clickhouse/tables/{cluster}/{shard}/{database}/{table} #849

Open Slach opened 7 months ago

supnobita commented 5 months ago

Hi @Slach, When i restored a table on a clickhouse cluster, I used --restore-database-mapping after that I saw clickhouse-backup create a different uuid for the same table on ZK, so that the table could not sync. Did it relate to this issue?

Slach commented 5 months ago

no this is a different issue, but could be related

usign {uuid} in replication path always bad idea from my perspective

do you mean restoring ReplicatedMergeTree on the different replicas with --restore-database-mapping replace {uuid} with different values on different replicas?

as a workaround you could change manually /var/lib/clickhouse/backup/backup_name/metadata/db/table.json and replace ReplicatedMergeTree('/path/{uuid}','{replica}') to ReplicatedMergeTree('/path/{shard}/{database}/{table}','{replica}')

supnobita commented 5 months ago

do you mean restoring ReplicatedMergeTree on the different replicas with --restore-database-mapping replace {uuid} with different values on different replicas? Yes ! And the new uuid(s) are different with the uuid in table.json in the backup metadata. That mean Clickhouse-backup create a new uuid, it didn't use the uuid of the backup.

I will try the workaround tmr. Thanks

Slach commented 5 months ago

you also could try to use

general:
 restore_metadata_on_cluster: "{cluster}"

after that ReplicatedMergeTree will create on cluster with the same UUID

it didn't use the uuid of the backup.

uuid is not related to backup this is clickhouse-server behavior when you created engine=ReplicatedMergeTree() without parameter the table will create with default_replica_path (which contains {uuid} and default_replica_path values

Slach commented 5 months ago

if you will use restore_metadata_on_cluster in this case you just need execute restore --schema only in one replica in whole cluster

and use restore --data only in first replica in each shard

supnobita commented 5 months ago

It worked !

Thank you.