[Bug]: collection not loaded, querynode restart, search/query delegator rpc error

[JamesBonddu]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version: 2.3.11
- Deployment mode(standalone or cluster): cluster
- MQ type(rocksmq, pulsar or kafka): pulsar   
- SDK version(e.g. pymilvus v2.0.0rc2): 2.3.6
- OS(Ubuntu or CentOS): Ubuntu20.0
- CPU/Memory: 521 core/1T
- GPU: 24G
- Others:

Current Behavior

milvus.md

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

[yanliang567]

@JamesBonddu please offer the full milvus pods logs for investigation. I looks like a known fix that was fixed in milvus 2.3.13, please retry on it. /assign @JamesBonddu /unassign

[JamesBonddu]

@JamesBonddu please offer the full milvus pods logs for investigation. I looks like a known fix that was fixed in milvus 2.3.13, please retry on it. /assign @JamesBonddu /unassign

i had updated milvus 2.4.1-gpu and still has error pulsar Too many requests to the same Bookie while reading.

docker-compose down 
docker-compose up -d

docker-compose.yaml.md

[yanliang567]

/assign @LoveEachDay /unassign

[yhmo]

Link to https://github.com/milvus-io/milvus/issues/33171

[xiaofan-luan]

@JamesBonddu how many collections do you have and how many nodes? I don't this is likely related to a pulsar issue

[xiaofan-luan]

1. when collection number is large, loading all collection from pulsar might be expensive. Solution:
2. reduce collection number might help
3. upgrade to 2.3.15 might help
4. try to increase pulsar cpu and memory might help