Not able to restore the database

[DipalPrajapati]

Error 1: [clickhouse][connect=21][stmt] close 2022/03/10 17:18:56.445324 error can't restore 'nonencry.non': error during filepath.Walk for part 'all_1_1_0': lstat /var/lib/clickhouse/backup/non_to_encry2/shadow/nonencry/non/all_1_1_0: no such file or directory

ls -ltr /var/lib/clickhouse/backup/non_to_encry2/shadow/nonencry/non/ total 24 -rw-r--r-- 1 root root 8192 Mar 10 17:18 default_all_1_1_0.tar -rw-r--r-- 1 root root 8192 Mar 10 17:18 default_all_2_2_0.tar -rw-r--r-- 1 root root 8192 Mar 10 17:18 default_all_3_3_0.tar

Error 2 : [clickhouse][connect=10][prepare] CREATE TABLE default.sample2 UUID '15d9e8f1-121e-4edf-ba10-f2f480573b10' (id UInt64) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 8192 [clickhouse][connect=10][send query] CREATE TABLE default.sample2 UUID '15d9e8f1-121e-4edf-ba10-f2f480573b10' (id UInt64) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 8192 [clickhouse][connect=10][query settings] log_queries=0&connect_timeout=300&receive_timeout=300&send_timeout=300 [clickhouse][connect=10][send external tables] count 0 [clickhouse][connect=10][process] <- exception 2022/03/10 16:54:33.164687 error can't create table default.sample2: code: 107, message: Cannot open file /var/lib/clickhouse/store/1e1/1e177f6c-69f0-4e91-9a16-dbbe4701c9dd/sample2.sql.tmp, errno: 2, strerror: No such file or directory after 1 times, please check your schema dependencies [root@master01 clickhouse-backup]#

[DipalPrajapati]

Still not working.

[Slach]

freeze_by_part not affects restore command, only create command do you attempt to restore on server where default.sample2 already exists?

looks really strange you try to restore with UUID '15d9e8f1-121e-4edf-ba10-f2f480573b10' but server try to get strange file 1e177f6c-69f0-4e91-9a16-dbbe4701c9dd/sample2.sql.tmp with different UUID

could you share grep -r '1e177f6c-69f0-4e91-9a16-dbbe4701c9dd' /var/lib/clickhouse/metadata ?

[DipalPrajapati]

Ok checking on that let me clean metadata and other store directory and recreate the db , as i am just testing the functionality yet
then I would be using this tool to move non encrypted data to encrypted disk. , can you check first error?

[Slach]

2022/03/10 17:18:56.445324 error can't restore 'nonencry.non': error during filepath.Walk for part 'all_1_1_0': lstat /var/lib/clickhouse/backup/non_to_encry2/shadow/nonencry/non/all_1_1_0: no such file or directory

Are you sure you successfully and complete upload and download your backup? look like your non_to_encry2 backup is broken

[Slach]

i am just testing the functionality could you describe your backup / restore workflow step by step?

[DipalPrajapati]

Yes , the backup is successful and ls -ltr /var/lib/clickhouse/backup/non_to_encry2 total 4 drwxr-xr-x 3 root root 22 Mar 10 17:18 shadow drwxr-xr-x 3 root root 22 Mar 10 17:18 metadata -rw-r--r-- 1 root root 743 Mar 10 17:18 metadata.json

[DipalPrajapati]

My scenarios:

1. Moving data from Non encrypted disk N1 to Encrypted disk E1 Have 2 standalone nodes.

Nonencrypted Disk have default path N1 -> /var/lib/clickhouse Encrypted Disk path E1 -> /var/lib/clickhouse/encrypted_one/

1. Creating remote backup on N1 , which uploads to home directory of E1.
2. I move files From E1's home directory to /var/lib/clickhouse/backup/
3. Execute Restore command on E1.

Which results in 2022/03/10 17:46:08.614506 error can't restore 'nonencry.non': error during filepath.Walk for part 'all_1_1_0': lstat /var/lib/clickhouse/backup/non_to_encry2/shadow/nonencry/non/all_1_1_0: no such file or directory

5. Moving data from Encrypted disk to Non encrypted disk

6. Creating remote backup on E1.

7. That uploads to home dir of N1.

8. Moving files from home to /var/lib/clickhouse/backup directory.

9. Execute restore command , which results in /var/lib/clickhouse/store/1e1/1e177f6c-69f0-4e91-9a16-dbbe4701c9dd/sample2.sql.tmp, errno: 2, strerror: No such file or directory after 1 times, please check your schema dependencies

Eventually all these would be performed on Containerized clickhouse cluster, once all of these works.

[DipalPrajapati]

In the first error i see , it tried to find all_1_1_0 but it has got default_all_1_1_0.tar If i do tar xvf tarfile manually , It give

[clickhouse][connect=31][hello] <- ClickHouse 22.2.54455 (Asia/Kolkata)
[clickhouse][connect=31][prepare] ALTER TABLE `nonencry`.`non` ATTACH PART 'all_1_3_1'
[clickhouse][connect=31][send query] ALTER TABLE `nonencry`.`non` ATTACH PART 'all_1_3_1'
[clickhouse][connect=31][query settings] log_queries=0&connect_timeout=300&receive_timeout=300&send_timeout=300
[clickhouse][connect=31][send external tables] count 0
[clickhouse][connect=31][process] <- exception
2022/03/10 18:07:16.245938 error can't attach partitions for table 'nonencry.non': code: 233, message: Detached part "all_1_3_1" not found
[root@master01 non]#
[root@master01 non]# ls -ltr
total 8
-rw-r--r-- 1 root       root       8192 Mar 10 18:04 default_all_1_3_1.tar
drwxr-xr-x 2 root       root        152 Mar 10 18:06 all_1_3_1
drwxr-x--- 3 clickhouse clickhouse   23 Mar 10 18:07 detached

[Slach]

Moving data from Non encrypted disk N1 to Encrypted disk E1

could you share SELECT * FROM system.storage_policy

why you use clickhouse-backup instead of use ALTER TABLE db.table MOVE PARTITION ... TO DISK 'E1'? look to https://clickhouse.com/docs/en/sql-reference/statements/alter/partition/#alter_move-partition

[Slach]

Creating remote backup on N1 , which uploads to home directory of E1.

weird decision, do you upload unencrypted data in TAR format to folder where clickhouse-server expect ecnrypted bin and mrk files? why?

[Slach]

is issue still relevant for you? or we could it close?