Altinity / clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.
https://altinity.com
Other
1.29k stars 226 forks source link

Fix wrong restoration of Materialized views with view name starting with digits #942

Closed praveenthuwat closed 4 months ago

praveenthuwat commented 4 months ago

Hi, If the materialized view names start with a digit and are enclosed in a backtick(`) during the view creation, the regex group capture for matching the TO database is missing. This further leads to the wrong restoration of materialized views, as the TO still points to the source database!

For example:- If we run the following restore command, clickhouse-backup restore_remote --rm -m default:praveen <backup-name>

The resulting materialized view will be the following. As you can see, the TO is still pointing to the source DB name, i.e., default instead of praveen)

CREATE MATERIALIZED VIEW praveen.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_mv_shard` TO default.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_mv_inner_shard` (

    `__created_at` AggregateFunction(max,
 DateTime),

    `__occurred_at` DateTime,

    `environment_id` UUID,

    `aggregation` String,

    `value` AggregateFunction(sum,
 Float64)
) AS
SELECT
    maxState(__created_at) AS __created_at,

    toStartOfFiveMinute(__occurred_at) AS __occurred_at,

    environment_id,

    aggregation,

    sumState(value) AS value
FROM praveen.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_shard` GROUP BY
    environment_id,

    aggregation,

    __occurred_at
ORDER BY
    environment_id ASC,

    aggregation ASC,

    __occurred_at ASC;

The expected one:-

CREATE MATERIALIZED VIEW praveen.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_mv_shard` TO praveen.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_mv_inner_shard` (

    `__created_at` AggregateFunction(max,
 DateTime),

    `__occurred_at` DateTime,

    `environment_id` UUID,

    `aggregation` String,

    `value` AggregateFunction(sum,
 Float64)
) AS
SELECT
    maxState(__created_at) AS __created_at,

    toStartOfFiveMinute(__occurred_at) AS __occurred_at,

    environment_id,

    aggregation,

    sumState(value) AS value
FROM praveen.`32da477b_0eb9_4aa6_bcbb_7f511649861e_v0_5_min_shard` GROUP BY
    environment_id,

    aggregation,

    __occurred_at
ORDER BY
    environment_id ASC,

    aggregation ASC,

    __occurred_at ASC;
coveralls commented 4 months ago

Pull Request Test Coverage Report for Build 9731076406

Details


Files with Coverage Reduction New Missed Lines %
pkg/backup/create.go 2 75.58%
pkg/backup/restore.go 2 67.08%
pkg/backup/backuper.go 2 65.12%
cmd/clickhouse-backup/main.go 2 77.19%
pkg/backup/list.go 2 63.44%
pkg/clickhouse/clickhouse.go 4 78.71%
pkg/storage/object_disk/object_disk.go 5 70.95%
pkg/status/status.go 9 70.2%
pkg/config/config.go 10 71.24%
pkg/storage/general.go 23 61.46%
<!-- Total: 265 -->
Totals Coverage Status
Change from base Build 9699341182: -2.0%
Covered Lines: 8552
Relevant Lines: 13048

💛 - Coveralls
Slach commented 4 months ago

thanks for contribution