[Bug]: Title: High Memory Consumption During Building MIlvus

[yellow-shine]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: any branch 
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

I am using 4 core cpu to build milvus. most of time, it just take 8G memory.

but occasionally build process could consumes more than 60 GB of RAM, which is significantly higher than expected. This excessive memory usage raises concerns about the feasibility of building Milvus on machines with limited memory resources.

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[yanliang567]

/assign @liliu-z /unassign