search

zilliztech / milvus-backup

Backup and restore tool for Milvus
Apache License 2.0
111 stars 38 forks source link

failed to check blob bucket exist #329

Open shaiktas opened 2 months ago

shaiktas commented 2 months ago

Hello Team, I have configured gcp bucket information in configs/backup.yaml as below:

minio:
  storageType: "gcp" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)

  address: "storage.googleapis.com"  ##localhost # Address of MinIO/S3
  port: 443   # Port of MinIO/S3
  useIAM: true # false

  bucketName: "my-bucket" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
  rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance
  backupBucketName: "backup" # Bucket name to store backup data. Backup data will store to 
  backupBucketName/backupRootPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath

backup:
  maxSegmentGroupSize: 2G

  parallelism: 
    backupCollection: 4
    copydata: 128
    restoreCollection: 2

  keepTempFiles: false

  gcPause:
    enable: true
    seconds: 7200
    address: http://localhost:9091

With above configs, when you i starting back up with command `./milvus-backup create test I am seeing below error: Pls review above configs and help me in fixing this issue

[2024/04/24 21:52:24.363 +00:00] [WARN] [storage/minio_chunk_manager.go:124] ["failed to check blob bucket exist->"] [address=storage.googleapis.com:443] [bucket=backup] [error="Get \"https://backup.storage.googleapis.com/?location\": proxyconnect tcp: dial tcp 10.112.66.68:443: i/o timeout"]

wayblink commented 1 month ago

I think it is largely related to gcp storage. Please make sure you can connect to gcp in your environment.