search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.29k stars 2.81k forks source link

[Bug]: load segment may cause long tSafe latency #35070

Open weiliu1031 opened 1 month ago

weiliu1031 commented 1 month ago

Is there an existing issue for this?

Environment

- Milvus version: 2.3/2.4/master
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

cause load segment will hold delete buffer lock to consume msg from stream, which may block process delete for a few seconds, and cause a peek in tSafe latency.

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

weiliu1031 commented 1 month ago

/assign

xiaofan-luan commented 1 month ago

I think this is not must? Once we split all the existing delete log there is no need to hold the delete log?

@XuanYang-cn

stale[bot] commented 22 hours ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.