search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
28.13k stars 2.71k forks source link

[Enhancement]: Avoid assign too much segment/channels to new querynode #34095

Open weiliu1031 opened 1 week ago

weiliu1031 commented 1 week ago

Is there an existing issue for this?

What would you like to be added?

When a new query node comes online, the segment_checker, channel_checker, and balance_checker simultaneously attempt to allocate segments to it. If this occurs during the execution of a load task and the distribution of the new query node hasn't been updated, the query coordinator may mistakenly view the new query node as empty. As a result, it assigns segments or channels to it, potentially overloading the new query node with more segments or channels than expected.

Why is this needed?

No response

Anything else?

No response

weiliu1031 commented 1 week ago

/assign

xiaofan-luan commented 1 week ago

for each balance step, we should set a limit(see only 5 segments can be balanced at the same time)