zilliztech / milvus-backup

Backup and restore tool for Milvus
Apache License 2.0
111 stars 38 forks source link

[Bug]: When I used Tencent Cloud to store Milvus backups, an error occurred #326

Open chenjinjun opened 2 months ago

chenjinjun commented 2 months ago

Current Behavior

An error occurred while executing the Milvus Backup check:

The specified bucket does not exist.

I have checked that my Key and Sercret are correct, and Mini Init passed normally. An error was reported when executing to ListWithPrefix. I once suspected that Key and Secret did not have permission, but I have repeatedly confirmed that they have full permission

My Config:

log:
  level: info 
  console: true
  file:
    rootPath: "logs/backup.log"

http:
  simpleResponse: true

milvus:
  address: localhost
  port: 19530
  authorizationEnabled: true
  tlsMode: 0
  user: "root"
  password: "Milvus"

minio:
  storageType: "tencent" 
  address: cos.xxx.myqcloud.com
  port: 443
  accessKeyID: xxx
  secretAccessKey: xxx
  useSSL: true 
  useIAM: false
  iamEndpoint: ""
  bucketName: "a-bucket"
  rootPath: "files" 
  backupBucketName: "xxx-xxx" 
  backupRootPath: "files"

backup:
  maxSegmentGroupSize: 2G
  parallelism: 
    backupCollection: 4
    copydata: 128
    restoreCollection: 2
  keepTempFiles: false
  gcPause:
    enable: true
    seconds: 7200
    address: http://localhost:9091

Expected Behavior

[root@VM-0-9-centos milvusback]# ./milvus-backup check 0.4.12 (Built on 2024-03-13T10:20:58Z from Git SHA 7b7c7b10c51e583085323cac86a5d4e16e9bf832) config:backup.yaml [2024/04/16 15:26:06.739 +08:00] [INFO] [logutil/logutil.go:165] ["Log directory"] [configDir=] [2024/04/16 15:26:06.739 +08:00] [INFO] [logutil/logutil.go:166] ["Set log file to "] [path=logs/backup.log] [2024/04/16 15:26:07.050 +08:00] [INFO] [storage/minio_chunk_manager.go:151] ["minio chunk manager init success."] [bucketname=xxx] [root=files] [2024/04/16 15:26:07.243 +08:00] [WARN] [storage/minio_chunk_manager.go:428] ["failed to list with prefix"] [prefix=files/] [error="The specified bucket does not exist."] Failed to connect to storage milvus path Milvus version: v2.3.12 Storage: milvus-bucket: a-bucket milvus-rootpath: files backup-bucket: aifa-1301323077 backup-rootpath: files The specified bucket does not exist.

Steps To Reproduce

No response

Environment

No response

Anything else?

No response

wayblink commented 1 month ago

So, is the two buckets exist?

chenjinjun commented 1 month ago

So, is the two buckets exist?

ofcourse

wayblink commented 1 month ago

@chenjinjun Well, the error message is a storage SDK buildin error. There must be something wrong about configs or environment. From the log, it points to the milvus bucket which is "a-bucket". Please check if it is the same with your Milvus config.