数据备份，通过API指定db_collections参数时，请求报错

liuylmr commented 2 days ago

Current Behavior

swagger ui的接口文档中，Create backup的接口文档有如下参数，支持指定数据库名称db_collections

{ "async": true, "backup_name": "string", "collection_names": [ "string" ], "db_collections": "string", "force": true, "gc_pause_address": "string", "gc_pause_enable": true, "gc_pause_seconds": 0, "meta_only": true, "requestId": "string" }

执行如下备份请求时 curl --location --request POST 'http://xxx/api/v1/create' --header 'Content-Type: application/json' --data '{ "async": false, "backup_name": "charai_hello_backup", "collection_names": ["hello_backup"], "db_collections": "char_ai"}

请求报错： {"requestId":"bcd37f6e-384e-11ef-acbe-0cda411d1468","code":3,"msg":"ReadMapCB: expect { or n, but found c, error found in #1 byte of ...|char_ai|..., bigger context ...|char_ai|...","data":{"id":"bcd37f6e-384e-11ef-acbe-0cda411d1468","state_code":3,"errorMessage":"ReadMapCB: expect { or n, but found c, error found in #1 byte of ...|char_ai|..., bigger context ...|char_ai|...","start_time":1719909629839,"name":"charai_hello_backup2","size":0,"milvus_version":"v2.3.10"}}

Expected Behavior

正常备份指定数据库名称下的collections

Steps To Reproduce

Milvus standalone: v2.3.10
Milvus_backup :v0.4.16

Environment

No response

Anything else?

No response

zhuwenxing commented 2 days ago

can you try with following command

curl --location --request POST 'http://xxx/api/v1/create' --header 'Content-Type: application/json' --data '{ "async": false, "backup_name": "charai_hello_backup", "collection_names": ["char_ai.hello_backup"]}

collection_name is set as db_name.collection_name

liuylmr commented 2 days ago

can you try with following command

curl --location --request POST 'http://xxx/api/v1/create' --header 'Content-Type: application/json' --data '{ "async": false, "backup_name": "charai_hello_backup", "collection_names": ["char_ai.hello_backup"]}

collection_name is set as db_name.collection_name

可以了，多谢。

liuylmr commented 1 day ago

can you try with following command

curl --location --request POST 'http://xxx/api/v1/create' --header 'Content-Type: application/json' --data '{ "async": false, "backup_name": "charai_hello_backup", "collection_names": ["char_ai.hello_backup"]}

collection_name is set as db_name.collection_name

When restoring backup data, using the interface http://xxx/api/v1/restore, how to specify the database name in the interface parameters I tried to specify in collection_names, but it didn't work. "collection_names": ["char.hello_milvus"],

zhuwenxing commented 1 day ago

The "collection_names" in the restore interface does not refer to what the restored collection will be named as, but rather it indicates which collections we want to restore from this backup.

zhuwenxing commented 1 day ago

payload = {"async": False, "backup_name": back_up_name, "collection_suffix":"bak"}

you can try with this payload. the collection will be restored to the same db and the collection name will be added a suffix

liuylmr commented 1 day ago

payload = {"async": False, "backup_name": back_up_name, "collection_suffix":"bak"}

you can try with this payload. the collection will be restored to the same db and the collection name will be added a suffix

I tried but it doesn't seem to work. The API request returns success, but the data hasn't actually been restored.

When restoring a backup without specifying a database name, it can restore the data correctly 。and the restore request returns as follow

{"requestId":"2f45c388-391e-11ef-a7f2-0cda411d1468","msg":"success","data":{"id":"restore_2024_07_03_09_25_27_689191275","start_time":1719998727,"collection_restore_tasks":[{"id":"2f4a22e9-391e-11ef-a7f2-0cda411d1468","state_code":2,"start_time":1719998727,"target_collection_name":"hello_milvus_bak","restored_size":145332,"to_restore_size":145332,"target_db_name":"default"}],"restored_size":0,"to_restore_size":145332}}

When restoring a backup with a specified database name, the data was not actually restored. and the restore request returns value that is different from the previous one. It seems that the restore task is missing。 {"requestId":"52614d2b-391e-11ef-a7f2-0cda411d1468","msg":"success","data":{"id":"restore_2024_07_03_09_26_26_598999254","start_time":1719998786,"restored_size":0,"to_restore_size":0}}%

zhuwenxing commented 1 day ago

https://github.com/zilliztech/milvus-backup/blob/0e539a0b86ab42afdcc0d21c0d19830b0b8f00e9/tests/testcases/test_restore_backup.py#L144C1-L189C80

We've covered this scenario in our test cases, and normally it shouldn't be a problem. If possible, could you upload the logs of milvus-backup so we can conduct a more in-depth analysis? Thank you.

liuylmr commented 1 day ago

https://github.com/zilliztech/milvus-backup/blob/0e539a0b86ab42afdcc0d21c0d19830b0b8f00e9/tests/testcases/test_restore_backup.py#L144C1-L189C80

We've covered this scenario in our test cases, and normally it shouldn't be a problem. If possible, could you upload the logs of milvus-backup so we can conduct a more in-depth analysis? Thank you.

yeah, I reviewed the test case and analyzed the logs, I have already found the issue. Thank you for the suggestion

zilliztech / milvus-backup