search

zilliztech / milvus-backup

Backup and restore tool for Milvus
Apache License 2.0
129 stars 46 forks source link

[Bug]: Only meta getting backup #190

Closed nileshchilka1 closed 4 months ago

nileshchilka1 commented 1 year ago

Current Behavior

Below is my config file, This config file only saving meta in minio

log: level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'. console: true file: rootPath: "logs/backup.log"

http: simpleResponse: true

milvus: address: localhost port: 19530 authorizationEnabled: false tlsMode: 0 user: "root" password: "Milvus"

minio: address: localhost port: 9000 accessKeyID: # accessKeyID of MinIO/S3 secretAccessKey: # MinIO/S3 encryption string useSSL: false # Access to MinIO/S3 with SSL useIAM: false cloudProvider: "aws" iamEndpoint: ""

bucketName: "milvus-cluster" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

backupBucketName: "milvus-bucket" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath

backup: maxSegmentGroupSize: 2G

Expected Behavior

Should save both meta and actual data

Steps To Reproduce

No response

Environment

Milvus deployed using Helm
milvus-backup 0.2.2
milvus_version 2.2.8

Anything else?

No response

wayblink commented 1 year ago

Please verify if bucketName and rootPath is the same with your Milvus cluster

nileshchilka1 commented 1 year ago

It is same I checked with helm values, Let me know how can I debug next

wayblink commented 1 year ago

Please change log level to debug, retry and upload your log

nileshchilka1 commented 1 year ago

I connected to minio using port forward from Kubernetes. By this we can confirm bucketName: "milvus-cluster" = and rootPath: "files"

WhatsApp Image 2023-09-11 at 12 13 23 PM

When I restored, below files I got in backupBucketName: "milvus-bucket"

WhatsApp Image 2023-09-11 at 12 14 45 PM

In Attu, I'm getting 0 in Approx. Entity Count

WhatsApp Image 2023-09-11 at 12 48 03 PM

nileshchilka1 commented 1 year ago

logs of restore

[2023/09/11 12:42:20.323 +05:30] [INFO] [logutil/logutil.go:165] ["Log directory"] [configDir=] [2023/09/11 12:42:20.325 +05:30] [INFO] [logutil/logutil.go:166] ["Set log file to "] [path=logs/backup.log] [2023/09/11 12:42:20.347 +05:30] [INFO] [core/backup_context.go:1021] ["receive RestoreBackupRequest"] [requestId=8c2de5f9-5072-11ee-8f93-c0185016d6ee] [backupName=fresh_backup] [collections="[]"] [CollectionSuffix=_recover] [CollectionRenames={}] [async=false] [bucketName=] [path=] [2023/09/11 12:42:20.348 +05:30] [DEBUG] [core/backup_context.go:64] ["Start Milvus client"] [endpoint=localhost:19530] [2023/09/11 12:42:20.738 +05:30] [DEBUG] [core/backup_context.go:88] ["Start minio client"] [address=localhost:9000] [bucket=milvus-cluster] [backupBucket=milvus-bucket] [2023/09/11 12:42:21.605 +05:30] [INFO] [storage/minio_chunk_manager.go:112] ["minio chunk manager init success."] [bucketname=milvus-bucket] [root=files] [2023/09/11 12:42:21.611 +05:30] [INFO] [core/backup_context.go:792] ["receive GetBackupRequest"] [requestId=8cf0514c-5072-11ee-8f93-c0185016d6ee] [backupName=fresh_backup] [backupId=] [bucketName=] [path=] [2023/09/11 12:42:23.389 +05:30] [INFO] [core/backup_context.go:868] ["finish GetBackupRequest"] [requestId=8cf0514c-5072-11ee-8f93-c0185016d6ee] [backupName=fresh_backup] [backupId=] [bucketName=] [path=] [resp="requestId:\"8cf0514c-5072-11ee-8f93-c0185016d6ee\" msg:\"success\" data:<id:\"8829b117-4d70-11ee-b1a0-c0185016d6ee\" state_code:BACKUP_SUCCESS start_time:1694085621307 end_time:1694085651251 name:\"fresh_backup\" backup_timestamp:1694085621307 collection_backups:<collection_id:443417093229978538 collection_name:\"pdf_walmart_test\" schema:<name:\"pdf_walmart_test\" fields:<fieldID:100 name:\"id\" is_primary_key:true description:\"Unique ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:101 name:\"doc_id\" description:\"Source document ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:102 name:\"text\" description:\"The embedding vector\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:103 name:\"embedding\" description:\"The embedding vector\" data_type:FloatVector type_params:<key:\"dim\" value:\"1536\" > > fields:<fieldID:104 name:\"node\" description:\"The node content\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > > shards_num:1 partition_backups:<partition_id:443417093229978539 partition_name:\"_default\" collection_id:443417093229978538 segment_backups:<segment_id:443417093243127763 collection_id:443417093229978538 partition_id:443417093229978539 num_of_rows:1 binlogs:<binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/0/443417093243130862\" log_size:533 > > binlogs:<fieldID:1 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/1/443417093243130863\" log_size:533 > > binlogs:<fieldID:100 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/100/443417093243130857\" log_size:682 > > binlogs:<fieldID:101 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/101/443417093243130858\" log_size:682 > > binlogs:<fieldID:102 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/102/443417093243130859\" log_size:18099 > > binlogs:<fieldID:103 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/103/443417093243130860\" log_size:6017 > > binlogs:<fieldID:104 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093243127763/104/443417093243130861\" log_size:5536 > > deltalogs:<> size:32082 > segment_backups:<segment_id:443417093241899747 collection_id:443417093229978538 partition_id:443417093229978539 num_of_rows:242 binlogs:<binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/0/443417093241899755\" log_size:1075 > > binlogs:<fieldID:1 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/1/443417093241899756\" log_size:726 > > binlogs:<fieldID:100 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/100/443417093241899750\" log_size:5918 > > binlogs:<fieldID:101 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/101/443417093241899751\" log_size:3490 > > binlogs:<fieldID:102 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/102/443417093241899752\" log_size:165985 > > binlogs:<fieldID:103 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/103/443417093241899753\" log_size:1375609 > > binlogs:<fieldID:104 binlogs:<log_path:\"files/insert_log/443417093229978538/443417093229978539/443417093241899747/104/443417093241899754\" log_size:36135 > > deltalogs:<> size:1588938 > size:1621020 load_state:\"Loaded\" > backup_timestamp:444094384439296 size:1621020 index_infos:<field_name:\"embedding\" index_name:\"_default_idx_103\" index_type:\"HNSW\" params:<key:\"index_type\" value:\"HNSW\" > params:<key:\"metric_type\" value:\"IP\" > params:<key:\"params\" value:\"{\\"M\\": 8, \\"efConstruction\\": 64}\" > > load_state:\"Loaded\" backup_physical_timestamp:1694085634 > collection_backups:<collection_id:443417093237547180 collection_name:\"hello_milvus_1\" schema:<name:\"hello_milvus_1\" description:\"hello_milvus is the simplest demo to introduce the APIs\" fields:<fieldID:100 name:\"pk\" is_primary_key:true data_type:VarChar type_params:<key:\"max_length\" value:\"100\" > > fields:<fieldID:101 name:\"random\" data_type:Double > fields:<fieldID:102 name:\"embeddings\" data_type:FloatVector type_params:<key:\"dim\" value:\"8\" > > > shards_num:1 partition_backups:<partition_id:443417093237547181 partition_name:\"_default\" collection_id:443417093237547180 segment_backups:<segment_id:443417093237547193 collection_id:443417093237547180 partition_id:443417093237547181 num_of_rows:3000 binlogs:<binlogs:<log_path:\"files/insert_log/443417093237547180/443417093237547181/443417093237547193/0/443417093237547216\" log_size:8133 > > binlogs:<fieldID:1 binlogs:<log_path:\"files/insert_log/443417093237547180/443417093237547181/443417093237547193/1/443417093237547217\" log_size:544 > > binlogs:<fieldID:100 binlogs:<log_path:\"files/insert_log/443417093237547180/443417093237547181/443417093237547193/100/443417093237547213\" log_size:6910 > > binlogs:<fieldID:101 binlogs:<log_path:\"files/insert_log/443417093237547180/443417093237547181/443417093237547193/101/443417093237547214\" log_size:27596 > > binlogs:<fieldID:102 binlogs:<log_path:\"files/insert_log/443417093237547180/443417093237547181/443417093237547193/102/443417093237547215\" log_size:91044 > > deltalogs:<fieldID:443417093237550321 binlogs:<log_path:\"files/delta_log/443417093237547180/443417093237547181/443417093237547193/443417093237550321\" log_size:765 > > size:134992 > size:134992 load_state:\"Loaded\" > backup_timestamp:444094386274304 size:134992 index_infos:<field_name:\"embeddings\" index_name:\"_default_idx_102\" index_type:\"IVF_FLAT\" params:<key:\"index_type\" value:\"IVF_FLAT\" > params:<key:\"metric_type\" value:\"L2\" > params:<key:\"params\" value:\"{\\"nlist\\": 128}\" > > load_state:\"Loaded\" backup_physical_timestamp:1694085641 > collection_backups:<collection_id:443417093231223060 collection_name:\"metadata_test\" schema:<name:\"metadata_test\" fields:<fieldID:100 name:\"id\" is_primary_key:true description:\"Unique ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:101 name:\"doc_id\" description:\"Source document ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:102 name:\"text\" description:\"The embedding vector\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:103 name:\"embedding\" description:\"The embedding vector\" data_type:FloatVector type_params:<key:\"dim\" value:\"1536\" > > fields:<fieldID:104 name:\"node\" description:\"The node content\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > > shards_num:1 partition_backups:<partition_id:443417093231223061 partition_name:\"_default\" collection_id:443417093231223060 segment_backups:<segment_id:443417093231223137 collection_id:443417093231223060 partition_id:443417093231223061 num_of_rows:8 binlogs:<binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/0/443417093231226226\" log_size:563 > > binlogs:<fieldID:1 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/1/443417093231226227\" log_size:542 > > binlogs:<fieldID:100 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/100/443417093231226221\" log_size:848 > > binlogs:<fieldID:101 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/101/443417093231226222\" log_size:733 > > binlogs:<fieldID:102 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/102/443417093231226223\" log_size:23111 > > binlogs:<fieldID:103 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/103/443417093231226224\" log_size:45832 > > binlogs:<fieldID:104 binlogs:<log_path:\"files/insert_log/443417093231223060/443417093231223061/443417093231223137/104/443417093231226225\" log_size:8408 > > deltalogs:<> size:80037 > size:80037 load_state:\"Loaded\" > backup_timestamp:444094387060736 size:80037 index_infos:<field_name:\"embedding\" index_name:\"_default_idx_103\" index_type:\"HNSW\" params:<key:\"index_type\" value:\"HNSW\" > params:<key:\"metric_type\" value:\"IP\" > params:<key:\"params\" value:\"{\\"M\\": 8, \\"efConstruction\\": 64}\" > > load_state:\"Loaded\" backup_physical_timestamp:1694085644 > collection_backups:<collection_id:443417093230654351 collection_name:\"memory_llama_index_v1\" schema:<name:\"memory_llama_index_v1\" fields:<fieldID:100 name:\"id\" is_primary_key:true description:\"Unique ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:101 name:\"doc_id\" description:\"Source document ID\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:102 name:\"text\" description:\"The embedding vector\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > fields:<fieldID:103 name:\"embedding\" description:\"The embedding vector\" data_type:FloatVector type_params:<key:\"dim\" value:\"1536\" > > fields:<fieldID:104 name:\"node\" description:\"The node content\" data_type:VarChar type_params:<key:\"max_length\" value:\"65535\" > > > shards_num:1 partition_backups:<partition_id:443417093230654352 partition_name:\"_default\" collection_id:443417093230654351 segment_backups:<segment_id:443417093230654423 collection_id:443417093230654351 partition_id:443417093230654352 num_of_rows:3 binlogs:<binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/0/443417093230657501\" log_size:551 > > binlogs:<fieldID:1 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/1/443417093230657502\" log_size:534 > > binlogs:<fieldID:100 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/100/443417093230657496\" log_size:733 > > binlogs:<fieldID:101 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/101/443417093230657497\" log_size:732 > > binlogs:<fieldID:102 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/102/443417093230657498\" log_size:16771 > > binlogs:<fieldID:103 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/103/443417093230657499\" log_size:17286 > > binlogs:<fieldID:104 binlogs:<log_path:\"files/insert_log/443417093230654351/443417093230654352/443417093230654423/104/443417093230657500\" log_size:5257 > > deltalogs:<> size:41864 > size:41864 load_state:\"Loaded\" > backup_timestamp:444094388109312 size:41864 index_infos:<field_name:\"embedding\" index_name:\"_default_idx_103\" index_type:\"HNSW\" params:<key:\"index_type\" value:\"HNSW\" > params:<key:\"metric_type\" value:\"IP\" > params:<key:\"params\" value:\"{\\"M\\": 8, \\"efConstruction\\": 64}\" > > load_state:\"Loaded\" backup_physical_timestamp:1694085648 > milvus_version:\"v2.2.8\" > "] [2023/09/11 12:42:23.408 +05:30] [INFO] [core/backup_context.go:1134] ["Collections to restore"] [collection_num=4] [2023/09/11 12:42:24.161 +05:30] [INFO] [core/backup_context.go:1219] ["executeRestoreBackupTask start"] [backup_name=fresh_backup] [backupBucketName=milvus-bucket] [backupPath=backup/fresh_backup] [2023/09/11 12:42:24.162 +05:30] [INFO] [core/backup_context.go:1258] ["start restore"] [collection_name=pdf_walmart_test_recover] [backupBucketName=milvus-bucket] [backupPath=backup/fresh_backup] [2023/09/11 12:42:24.655 +05:30] [INFO] [core/backup_context.go:1299] ["create collection"] [collectionName=pdf_walmart_test_recover] [2023/09/11 12:42:24.847 +05:30] [INFO] [core/backup_context.go:1316] ["create partition"] [collectionName=pdf_walmart_test_recover] [partitionName=_default] [2023/09/11 12:42:24.848 +05:30] [INFO] [core/backup_context.go:1457] [getBackupPartitionPaths] [bucketName=milvus-bucket] [backupPath=backup/fresh_backup] [partitionID=443417093229978539] [2023/09/11 12:42:25.047 +05:30] [DEBUG] [core/backup_context.go:1390] ["execute bulk insert"] [collection=pdf_walmart_test_recover] [partition=_default] [files="[backup/fresh_backup/binlogs/insert_log/443417093229978538/443417093229978539/,]"] [endTime=444094384439296] [2023/09/11 12:42:25.455 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960216] [state=5] [state="{\"ID\":443417093244960216,\"State\":5,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"pdf_walmart_test_recover\",\"end_ts\":\"444094384439296\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093229978538/443417093229978539/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"90\"},\"CollectionID\":443417093244960210,\"SegmentIDs\":null,\"CreateTs\":1694416342}"] [progress=90] [currentTimestamp=1694416345] [lastUpdateTime=1694416345] [2023/09/11 12:42:30.645 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960216] [state=6] [state="{\"ID\":443417093244960216,\"State\":6,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"pdf_walmart_test_recover\",\"end_ts\":\"444094384439296\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093229978538/443417093229978539/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"100\"},\"CollectionID\":443417093244960210,\"SegmentIDs\":null,\"CreateTs\":1694416342}"] [progress=100] [currentTimestamp=1694416350] [lastUpdateTime=1694416345] [2023/09/11 12:42:30.645 +05:30] [INFO] [core/backup_context.go:1239] ["finish restore collection"] [collection_name=pdf_walmart_test_recover] [2023/09/11 12:42:30.646 +05:30] [INFO] [core/backup_context.go:1258] ["start restore"] [collection_name=hello_milvus_1_recover] [backupBucketName=milvus-bucket] [backupPath=backup/fresh_backup] [2023/09/11 12:42:31.135 +05:30] [INFO] [core/backup_context.go:1299] ["create collection"] [collectionName=hello_milvus_1_recover] [2023/09/11 12:42:31.325 +05:30] [INFO] [core/backup_context.go:1316] ["create partition"] [collectionName=hello_milvus_1_recover] [partitionName=_default] [2023/09/11 12:42:31.326 +05:30] [INFO] [core/backup_context.go:1457] [getBackupPartitionPaths] [bucketName=milvus-bucket] [backupPath=backup/fresh_backup] [partitionID=443417093237547181] [2023/09/11 12:42:31.543 +05:30] [DEBUG] [core/backup_context.go:1390] ["execute bulk insert"] [collection=hello_milvus_1_recover] [partition=_default] [files="[backup/fresh_backup/binlogs/insert_log/443417093237547180/443417093237547181/,]"] [endTime=444094386274304] [2023/09/11 12:42:31.951 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960259] [state=5] [state="{\"ID\":443417093244960259,\"State\":5,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"hello_milvus_1_recover\",\"end_ts\":\"444094386274304\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093237547180/443417093237547181/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"90\"},\"CollectionID\":443417093244960253,\"SegmentIDs\":null,\"CreateTs\":1694416348}"] [progress=90] [currentTimestamp=1694416351] [lastUpdateTime=1694416351] [2023/09/11 12:42:37.160 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960259] [state=6] [state="{\"ID\":443417093244960259,\"State\":6,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"hello_milvus_1_recover\",\"end_ts\":\"444094386274304\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093237547180/443417093237547181/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"100\"},\"CollectionID\":443417093244960253,\"SegmentIDs\":null,\"CreateTs\":1694416348}"] [progress=100] [currentTimestamp=1694416357] [lastUpdateTime=1694416351] [2023/09/11 12:42:37.161 +05:30] [INFO] [core/backup_context.go:1239] ["finish restore collection"] [collection_name=hello_milvus_1_recover] [2023/09/11 12:42:37.161 +05:30] [INFO] [core/backup_context.go:1258] ["start restore"] [collection_name=metadata_test_recover] [backupBucketName=milvus-bucket] [backupPath=backup/fresh_backup] [2023/09/11 12:42:37.654 +05:30] [INFO] [core/backup_context.go:1299] ["create collection"] [collectionName=metadata_test_recover] [2023/09/11 12:42:37.850 +05:30] [INFO] [core/backup_context.go:1316] ["create partition"] [collectionName=metadata_test_recover] [partitionName=_default] [2023/09/11 12:42:37.851 +05:30] [INFO] [core/backup_context.go:1457] [getBackupPartitionPaths] [bucketName=milvus-bucket] [backupPath=backup/fresh_backup] [partitionID=443417093231223061] [2023/09/11 12:42:38.075 +05:30] [DEBUG] [core/backup_context.go:1390] ["execute bulk insert"] [collection=metadata_test_recover] [partition=_default] [files="[backup/fresh_backup/binlogs/insert_log/443417093231223060/443417093231223061/,]"] [endTime=444094387060736] [2023/09/11 12:42:38.502 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960299] [state=5] [state="{\"ID\":443417093244960299,\"State\":5,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"metadata_test_recover\",\"end_ts\":\"444094387060736\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093231223060/443417093231223061/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"90\"},\"CollectionID\":443417093244960292,\"SegmentIDs\":null,\"CreateTs\":1694416355}"] [progress=90] [currentTimestamp=1694416358] [lastUpdateTime=1694416358] [2023/09/11 12:42:43.710 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960299] [state=6] [state="{\"ID\":443417093244960299,\"State\":6,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"metadata_test_recover\",\"end_ts\":\"444094387060736\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093231223060/443417093231223061/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"100\"},\"CollectionID\":443417093244960292,\"SegmentIDs\":null,\"CreateTs\":1694416355}"] [progress=100] [currentTimestamp=1694416363] [lastUpdateTime=1694416358] [2023/09/11 12:42:43.712 +05:30] [INFO] [core/backup_context.go:1239] ["finish restore collection"] [collection_name=metadata_test_recover] [2023/09/11 12:42:43.712 +05:30] [INFO] [core/backup_context.go:1258] ["start restore"] [collection_name=memory_llama_index_v1_recover] [backupBucketName=milvus-bucket] [backupPath=backup/fresh_backup] [2023/09/11 12:42:44.212 +05:30] [INFO] [core/backup_context.go:1299] ["create collection"] [collectionName=memory_llama_index_v1_recover] [2023/09/11 12:42:44.412 +05:30] [INFO] [core/backup_context.go:1316] ["create partition"] [collectionName=memory_llama_index_v1_recover] [partitionName=_default] [2023/09/11 12:42:44.412 +05:30] [INFO] [core/backup_context.go:1457] [getBackupPartitionPaths] [bucketName=milvus-bucket] [backupPath=backup/fresh_backup] [partitionID=443417093230654352] [2023/09/11 12:42:44.626 +05:30] [DEBUG] [core/backup_context.go:1390] ["execute bulk insert"] [collection=memory_llama_index_v1_recover] [partition=_default] [files="[backup/fresh_backup/binlogs/insert_log/443417093230654351/443417093230654352/,]"] [endTime=444094388109312] [2023/09/11 12:42:45.267 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960339] [state=5] [state="{\"ID\":443417093244960339,\"State\":5,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"memory_llama_index_v1_recover\",\"end_ts\":\"444094388109312\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093230654351/443417093230654352/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"90\"},\"CollectionID\":443417093244960332,\"SegmentIDs\":null,\"CreateTs\":1694416361}"] [progress=90] [currentTimestamp=1694416365] [lastUpdateTime=1694416364] [2023/09/11 12:42:50.493 +05:30] [INFO] [core/backup_context.go:1422] ["bulkinsert task state"] [id=443417093244960339] [state=6] [state="{\"ID\":443417093244960339,\"State\":6,\"RowCount\":0,\"IDList\":null,\"Infos\":{\"backup\":\"true\",\"collection\":\"memory_llama_index_v1_recover\",\"end_ts\":\"444094388109312\",\"failed_reason\":\"\",\"files\":\"backup/fresh_backup/binlogs/insert_log/443417093230654351/443417093230654352/,\",\"partition\":\"_default\",\"persist_cost\":\"0.01\",\"progress_percent\":\"100\"},\"CollectionID\":443417093244960332,\"SegmentIDs\":null,\"CreateTs\":1694416361}"] [progress=100] [currentTimestamp=1694416370] [lastUpdateTime=1694416365] [2023/09/11 12:42:50.494 +05:30] [INFO] [core/backup_context.go:1239] ["finish restore collection"] [collection_name=memory_llama_index_v1_recover]

Zhaikuku commented 1 year ago

I also encountered a similar problem, my problem is: backup in A milvus, and then use A milvus backup to restore to B milvus, there is no data.

f245661189aecf31641694853826462

wayblink commented 12 months ago

@Zhaikuku @nileshchilka1 Sorry to reply quite late. Does the problem solved. You can use the latest 0.4.0 version and try ./milvus-backup check command help to see if the connection is right. If you still get empty collection after restore. Please upload your back.yaml and log, Thanks

axiangcoding commented 9 months ago

Same issue. Should i open a new issue?