I created nomad job for backup and upload clickhouse backup to S3, but after upload dump there no shadow folder with data.
My clickhouse-backup config:
2022/08/05 15:26:04.026255 info SELECT name, engine FROM system.databases WHERE name NOT IN ('system', 'INFORMATION_SCHEMA', 'information_schema')
2022/08/05 15:26:04.037118 info SHOW CREATE DATABASE `test`
2022/08/05 15:26:04.047324 info SHOW CREATE DATABASE `default`
2022/08/05 15:26:04.055358 info SELECT count() FROM system.settings WHERE name = 'show_table_uuid_in_table_create_query_if_not_nil'
2022/08/05 15:26:04.066642 info SELECT name FROM system.databases WHERE engine IN ('MySQL','PostgreSQL')
2022/08/05 15:26:04.074037 info SELECT countIf(name='data_path') is_data_path_present, countIf(name='data_paths') is_data_paths_present, countIf(name='uuid') is_uuid_present, countIf(name='create_table_query') is_create_table_query_present, countIf(name='total_bytes') is_total_bytes_present FROM system.columns WHERE database='system' AND table='tables'
2022/08/05 15:26:04.082602 info SELECT database, name, engine , data_paths , uuid , create_table_query , coalesce(total_bytes, 0) AS total_bytes FROM system.tables WHERE is_temporary = 0 SETTINGS show_table_uuid_in_table_create_query_if_not_nil=1
2022/08/05 15:26:04.113774 info SELECT sum(bytes_on_disk) as size FROM system.parts WHERE active AND database='test' AND table='environmental_zone_report' GROUP BY database, table
2022/08/05 15:26:04.128393 info SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
2022/08/05 15:26:04.140865 info SELECT * FROM system.disks;
2022/08/05 15:26:04.145439 info ALTER TABLE `test`.`environmental_sensor_data` FREEZE WITH NAME '1172c11669bc4a829363db32a00980a5';
2022/08/05 15:26:04.154697 info done backup=2022-08-05T07-59-40 operation=create table=testenvironmental_sensor_data
2022/08/05 15:26:04.155356 info ALTER TABLE `test`.`environmental_zone_report` FREEZE WITH NAME 'fe93966fe50c460584c9e63e29b11a65';
2022/08/05 15:26:04.163227 info done backup=2022-08-05T07-59-40 operation=create table=test.environmental_zone_report
2022/08/05 15:26:04.163278 info ALTER TABLE `test`.`schema_versions` FREEZE WITH NAME 'e27462da9a9d48ebbbbc2869057dbcba';
2022/08/05 15:26:04.171123 info done backup=2022-08-05T07-59-40 operation=create table=test.schema_versions
2022/08/05 15:26:04.172056 info SELECT toUInt8(count()) udf_presents FROM system.columns WHERE database='system' AND table='functions' AND name='create_query'
2022/08/05 15:26:04.180764 info SELECT value FROM `system`.`build_options` where name='VERSION_DESCRIBE'
2022/08/05 15:26:04.188804 info done backup=2022-08-05T07-59-40 duration=171ms operation=create
clickhouse-backup require direct access to the same /var/lib/clickhouse as clickhouse-server
use the same volumes for clickhouse-backup and clickhouse-server inside Nomad
Hi,
I created nomad job for backup and upload clickhouse backup to S3, but after upload dump there no
shadow
folder with data. My clickhouse-backup config:Output from batch job:
S3 bucket structure:
Maybe is there some option I forgot to enable?
I also tried to restore data from S3 bucket as is it, but had no luck. The old data wasn't pulled up.