Altinity / clickhouse-backup

Tool for easy backup and restore for ClickHouse® using object storage for backup files.
https://altinity.com
Other
1.29k stars 227 forks source link

URL is getting malformed while running restore_remote command with azure_blob_store #931

Closed rishi-cds closed 5 months ago

rishi-cds commented 5 months ago

We have setup clickhouse-backup CLI in our Clickhouse installation in Azure Kubernetes Service. after that using Clickhosue bakcup CLI we were able to take the backup of our Clickhouse cluster and files were uploaded successfully in the Azure container.

While restoring the same backup with same config, we are getting below error.

2024/06/07 13:03:12.276382 error one of restoreDataRegular go-routine return error: can't restore object_disk server-side copy data parts 'default.<table-name>': -> github.com/Azure/azure-pipeline-go/pipeline.NewError, /home/runner/go/pkg/mod/github.com/!azure/azure-pipeline-go@v0.2.3/pipeline/error.go:157
HTTP request failed

Put "https://<account-name>.blob.<storage-account-name>.blob.core.windows.net/<container-name-here>?restype=container&timeout=14401": dial tcp: lookup <storage-account-name>.blob.<storage-account-name>.blob.core.windows.net on 10.0.0.10:53: no such host

looks like it is appending an additional account_name.blob in the URL.

here is the config file we used.

general:
    remote_storage: azblob
    log_level: info
    allow_empty_backups: false
    upload_by_part: true
    download_by_part: true

clickhouse:
    username: "<ch-user>"
    password: "<ch-password>"
    host: localhost
    port: 9000
    disk_mapping: {}

azblob:
    account_name: "<storage-account-name>"
    use_managed_identity: true
    container: "<container-name>"
    debug: true
Slach commented 5 months ago

actually it looks like DNS issue and maybe not related to clickhouse-backup itself hostname created inside SDK

need more context for reproducing

do you have object disks data?

could you share?

SELECT * FROM system.disks
SELECT * FROM system.storage_policies
Slach commented 5 months ago

also could you check log erorrs near AZBLOB->CopyObject

Slach commented 5 months ago

is any news from your side?

Slach commented 5 months ago

closing due issue reporter inactivity