search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.77k stars 2.85k forks source link

[Enhancement]: Use BatchPkExist to reduce bloom filter func call cost #33610

Closed weiliu1031 closed 2 months ago

weiliu1031 commented 4 months ago

Is there an existing issue for this?

What would you like to be added?

During deletion, Milvus will attempt to predict the primary key’s segment using a bloom filter. For each primary key, it needs to make predictions across all segments. Consequently, this results in segment_num * delete_pk_num bloom filter accesses. For instance, if we delete 1 million records in a collection with 100 segments, it would lead to 1 billion bloom filter accesses. Therefore, ignoring the function call is not feasible.

Why is this needed?

No response

Anything else?

No response

weiliu1031 commented 4 months ago

/assign

stale[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.