Milvus data when migration not correct at sqlite

[binhvcom]

1. Error:

• Collection name, partition, index, none of them are migrate to the new host
• Old Host:

• New Host: After migrations:

2. How to reproduce:

• Run pymilvusdm==1.0 and pymilvus==1.0.1, migrate data of milvus server 1.0.0
• Run both on M2M.yaml, H2M.yaml and M2H.yaml
• Data migrate success but missing improtant data
• i have tested on both append and overwrite
• data for test

  from milvus import Milvus, IndexType, MetricType, Status
  milvus = Milvus(host='milvusv2.local', port='19530')
  param = {'collection_name':'test01', 'dimension':256, 'index_file_size':1024, 'metric_type':MetricType.L2}
  milvus.create_collection(param)
  milvus.create_partition('test01', 'tag01')
  import random
  vectors = [[random.random() for _ in range(256)] for _ in range(20)]
  vector_ids = [id for id in range(20)]
  milvus.insert(collection_name='test01', records=vectors, ids=vector_ids)
  milvus.insert('test01', vectors, partition_tag="tag01")
  ivf_param = {'nlist': 16384}
  milvus.create_index('test01', IndexType.IVF_FLAT, ivf_param)

• Docker-compose

services: milvus: image: 'milvusdb/milvus:1.0.0-cpu-d030521-1ea92e' hostname: milvus.local networks: binhbtn: ipv4_address: 172.23.0.3 volumes:

• /tmp/db:/var/lib/milvus/db
• /tmp/logs:/var/lib/milvus/logs
• /tmp/wal:/var/lib/milvus/wal milvusv2: image: 'milvusdb/milvus:1.0.0-cpu-d030521-1ea92e' hostname: milvusv2.local networks: binhbtn: ipv4_address: 172.23.0.4 volumes:
• /tmp/2/db:/var/lib/milvus/db
• /tmp/2/logs:/var/lib/milvus/logs
• /tmp/2/wal:/var/lib/milvus/wal python37: image: 'python:3.7.13' tty: true networks: binhbtn: ipv4_address: 172.23.0.5 volumes:
• /tmp/2/db:/var/lib/milvus/db
• /tmp/2/logs:/var/lib/milvus/logs
• /tmp/2/wal:/var/lib/milvus/wal
• /tmp/db:/var/lib/milvus/dest/db
• /tmp/logs:/var/lib/milvus/dest/logs
• /tmp/wal:/var/lib/milvus/dest/wal depends_on:
• milvus
• milvusv2

networks: binhbtn: driver: bridge ipam: config:

• subnet: 172.23.0.0/16

3. Config

• M2M.yaml

  M2M:  milvus_version: 1.0.0
  source_milvus_path: '/var/lib/milvus'
  mysql_parameter:
  source_collection:
  test01:
  dest_host: 'milvus.local'
  dest_port: 19530
  mode: 'overwrite' 

• H2M.yaml

  H2M:
  milvus_version: 1.x
  data_path:
  data_dir: '/var/lib/milvus/backup'
  dest_host: '172.23.0.3'
  dest_port: 19530
  mode: 'overwrite'
  dest_collection_name: 'test01'
  dest_partition_name: 'tag01'
  collection_parameter:
  dimension:
  index_file_size:
  metric_type:

• M2H.yaml

  M2H:
  milvus_version: 1.0.0
  source_milvus_path: '/var/lib/milvus'
  mysql_parameter:
  source_collection:
  test01:
  data_dir: '/var/lib/milvus/backup'