search

milvus-io / milvus-tools

A data migration tool for Milvus.
Apache License 2.0
71 stars 21 forks source link

Error occurred when migrating from Milvus to HDF5 #40

Open zhangcong2711 opened 2 years ago

zhangcong2711 commented 2 years ago

An error encountered when using DM to migrate data from Milvus to HDF5.

22-06-07 16:21:30,979 ｜ INFO ｜ milvus_to_hdf5.py ｜ read_milvus_data ｜ 49 ｜ Ready to read all data of collection: video_fingerprint/partitions: [None] 0%| | 0/1 [00:03<?, ?it/s] 2022-06-07 16:21:34,230 ｜ ERROR ｜ milvus_to_hdf5.py ｜ read_milvus_data ｜ 56 ｜ Error with: name 'delids' is not defined

Have tried different versions of milvusdm (1.0, 2.0) and the results are same. And it happened with another milvus server.

Version of milvusdm 2.0 Version of milvus 1.1.1

Configuration shown below ( M2H.yaml )

M2H: milvus_version: 1.1.1 source_milvus_path: '/data1/milvus_1.x_uni_video' mysql_parameter: host: '127.0.0.1' user: 'root' port: 3376 password: 'xxxxxxxxxxxx' database: 'milvus' source_collection: video_fingerprint: data_dir: '/data1/milvus_migration/uni_video' mode: 'overwrite'

shiyu22 commented 2 years ago

I think this issue is fixed in the main branch but it hasn't published, so you can follow it to reinstall pymilvusdm:

$ git clone https://github.com/milvus-io/milvus-tools.git
$ cd milvus-tools
$ echo "y" | pip uninstall pymilvusdm && python setup.py install