FR: support encrypted disks properly

[goto217]

I config encrypted disks, i backup clickhouse datas, two dirs are automatically generated by clickhouse-backup

• /var/lib/clickhouse/backup/2021-09-07T07-37-23/metadata/cdr/code_master.json
• /var/lib/clickhouse/encrypted/backup/2021-09-07T07-37-23/shadow/cdr/code_master/disk_encrypted/201807_1_1_0

I migrated the two dirs to the corresponding directory of the second clickhouse server, and execute restore by clickhouse-backup. Report the following error:

2021/09/08 09:06:44 error can't attach partitions for table 'cdr.code_master': code: 233, message: Detached part "201807_1_1_0" not found

how to do when i backup encrypted disks for clickhouse?

ENC CONFIG:

    <storage_configuration>
        <disks>
            <disk_unenc>
                <type>local</type>
                <path>/var/lib/clickhouse/</path>
            </disk_unenc>
            <disk_encrypted>
                <type>encrypted</type>
                <disk>disk_unenc</disk>
        <path>encrypted/</path>
                <algorithm>AES_128_CTR</algorithm>
                <key_hex id="0">ddb32656b5d583fb9994fade9c0707f9</key_hex>
                <key_hex id="1">8ccc4a1d35e8fc020fb1af9301cbf8f0</key_hex>
                <current_key_id>1</current_key_id>
            </disk_encrypted>
        </disks>
        <policies>
            <data_enc> 
                <volumes>
                    <single>
                        <disk>disk_encrypted</disk>
                    </single>
                </volumes>
            </data_enc>
        </policies>
    </storage_configuration>

[Slach]

@goto217 clickhouse-backup is supports encrypted disk

look like your problem should fixed by https://github.com/AlexAkulov/clickhouse-backup/pull/258 try to change <disk_encrypted><path> to /var/lib/clickhouse_encrypted to avoid nested paths

[goto217]

@goto217 clickhouse-backup is supports encrypted disk

look like your problem should fixed by #258 try to change <disk_encrypted><path> to /var/lib/clickhouse_encrypted to avoid nested paths

@Slach

I modify my clickhouse config:

    <storage_configuration>
        <disks>
            <disk_unenc>
                <type>local</type>
                <path>/var/lib/clickhouse/</path>
            </disk_unenc>
            <disk_encrypted>
                <type>encrypted</type>
                <disk>disk_unenc</disk>
        <path>/var/lib/ch-encrypted-data/</path>
                <algorithm>AES_128_CTR</algorithm>
                <key_hex id="0">ddb32656b5d583fb9994fade9c0707f9</key_hex>
                <key_hex id="1">8ccc4a1d35e8fc020fb1af9301cbf8f0</key_hex>
                <current_key_id>1</current_key_id>
            </disk_encrypted>
        </disks>
        <policies>
            <data_enc> 
                <volumes>
                    <single> 
                        <disk>disk_encrypted</disk>
                    </single>
                </volumes>
            </data_enc>
        </policies>
    </storage_configuration>

two dirs are automatically generated by clickhouse-backup

• /var/lib/clickhouse/backup/2021-09-08T06-59-15/metadata/cdr/code_master.json
• /var/lib/ch-encrypted-data/backup/2021-09-08T06-59-15/shadow/cdr/code_master/disk_encrypted/201807_1_1_0

execute restore by clickhouse-backup. still report the following error:

2021/09/08 15:02:27 error can't attach partitions for table 'cdr.code_master': code: 233, message: Detached part "201807_1_1_0" not found

generate "detached/201807_1_1_0" after restore in same level directory of execution file [clickhouse-backup] , and has .idx, .dat ... files in "detached/201807_1_1_0"

Is wrong for my clickhouse-backup config ?

clickhouse:
  username: default
  password: "123456"
  host: localhost
  port: 9000
  disk_mapping: {
    "disk_unenc":"/var/lib/clickhouse/",
    "disk_encrypted":"/var/lib/ch-encrypted-data/"
  }
  skip_tables:
  - system.*
  timeout: 5m
  freeze_by_part: false
  secure: false
  skip_verify: false
  sync_replicated_tables: true
  skip_sync_replica_timeouts: true
  log_sql_queries: false

258 has resovled ?

my clickhouse-backup last version:

Version:     1.0.0
Git Commit:  37f3bd78adec2aadc1de10f9323fe426a5e12dc4
Build Date:  2021-06-16

[Slach]

@obazna your config look OK.

look like clickhouse-backup doesn't support encrypted disks right now (it's a new functionality available from 21.9)

I need to deeply figure out how encrypted disk exactly works, currently, it looks like an abstraction layer on top of the real disk which implements encryption on file level inside IDisk interface

let's wait when 21.9 will released in docker hub and I will try to reproduce your behavior