[Bug]: Currently, there are three nodes. After the all data is written to the default partition, there is a significant difference in memory usage among different nodes.

[zk903956028]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:
- Deployment mode(cluster):
- MQ type(rocksmq, pulsar or kafka):    
- OS(CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

The cluster version is installed on three nodes with the version number 2.1.3. After the data is written, one node has high memory usage, while the other two nodes have low memory usage. This may affect the query efficiency.

Expected Behavior

Distribute data evenly across three different nodes to balance memory usage.

Steps To Reproduce

Below is the memory usage situation of the three nodes, it's obvious that Node devtest2 has a larger memory usage

[root@devtest1 etcd]# free -h
              total        used        free      shared  buff/cache   available
Mem:           376G         31G         16G        938M        329G        341G
Swap:            0B          0B          0B

[root@devtest2 /]# free -h
              total        used        free      shared  buff/cache   available
Mem:           376G        237G        7.5G        731M        131G        135G
Swap:            0B          0B          0B

[root@devtest3 ~]# free -h
              total        used        free      shared  buff/cache   available
Mem:           376G         26G         71G        210M        278G        347G
Swap:            0B          0B          0B

Milvus Log

无

Anything else?

无

[xiaofan-luan]

1. please try 2.2.14 and many problems has been fixed.
2. it might happen if data is not indexed. you can use birdwatcher to verify it , and add more indexnode would help this issue

[yanliang567]

agreed with xiaofan, @zk903956028 please retry on latest milvus release v2.2.14 or v2.3.0, which fixed many balance issues. /assign @zk903956028 /unassign

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.