milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
31.03k stars 2.95k forks source link

enhance: speed up search iterator stage 1 #37947

Open PwzXxm opened 1 week ago

PwzXxm commented 1 week ago

issue: #37548

sre-ci-robot commented 1 week ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: PwzXxm To complete the pull request process, please assign xiaofan-luan after the PR has been reviewed. You can assign the PR to them by writing /assign @xiaofan-luan in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/milvus-io/milvus/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
mergify[bot] commented 1 week ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

codecov[bot] commented 1 week ago

Codecov Report

Attention: Patch coverage is 93.00000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 82.94%. Comparing base (61f73d5) to head (9f6b887). Report is 2 commits behind head on master.

:exclamation: Current head 9f6b887 differs from pull request most recent head e7d1366

Please upload reports for the commit e7d1366 to get more accurate results.

Files with missing lines Patch % Lines
internal/proxy/search_util.go 89.39% 5 Missing and 2 partials :warning:
Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/graphs/tree.svg?width=650&height=150&src=pr&token=XdCbzGJk2E&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) ```diff @@ Coverage Diff @@ ## master #37947 +/- ## ========================================== + Coverage 81.06% 82.94% +1.88% ========================================== Files 1369 1080 -289 Lines 190968 165569 -25399 ========================================== - Hits 154812 137335 -17477 + Misses 30661 22736 -7925 - Partials 5495 5498 +3 ``` | [Components](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | Coverage Δ | | |---|---|---| | [Client](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `73.67% <ø> (ø)` | | | [Core](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `∅ <ø> (∅)` | | | [Go](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `83.20% <93.00%> (+<0.01%)` | :arrow_up: | | [Files with missing lines](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | Coverage Δ | | |---|---|---| | [internal/proxy/proxy.go](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?src=pr&el=tree&filepath=internal%2Fproxy%2Fproxy.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-aW50ZXJuYWwvcHJveHkvcHJveHkuZ28=) | `70.95% <100.00%> (+0.62%)` | :arrow_up: | | [internal/proxy/task.go](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?src=pr&el=tree&filepath=internal%2Fproxy%2Ftask.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-aW50ZXJuYWwvcHJveHkvdGFzay5nbw==) | `87.03% <ø> (ø)` | | | [internal/proxy/task\_search.go](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?src=pr&el=tree&filepath=internal%2Fproxy%2Ftask_search.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-aW50ZXJuYWwvcHJveHkvdGFza19zZWFyY2guZ28=) | `76.50% <100.00%> (+0.55%)` | :arrow_up: | | [internal/proxy/search\_util.go](https://app.codecov.io/gh/milvus-io/milvus/pull/37947?src=pr&el=tree&filepath=internal%2Fproxy%2Fsearch_util.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-aW50ZXJuYWwvcHJveHkvc2VhcmNoX3V0aWwuZ28=) | `82.31% <89.39%> (+0.94%)` | :arrow_up: | ... and [311 files with indirect coverage changes](https://app.codecov.io/gh/milvus-io/milvus/pull/37947/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)
PwzXxm commented 4 days ago

rerun cpp-unit-test

mergify[bot] commented 4 days ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] commented 4 days ago

@PwzXxm E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] commented 3 days ago

@PwzXxm go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] commented 3 days ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] commented 2 days ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] commented 2 days ago

@PwzXxm E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify[bot] commented 7 hours ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

PwzXxm commented 4 hours ago

/hold

PwzXxm commented 4 hours ago

/unhold Rename iterator token to iterator id

mergify[bot] commented 3 hours ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify[bot] commented 3 hours ago

@PwzXxm E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

MrPresent-Han commented 2 hours ago

/lgtm

PwzXxm commented 2 hours ago

/hold wait for inputs from @czs007 about backwards compatibility. New SDK -> Old Milvus Server

PwzXxm commented 1 hour ago

/unhold If the user uses New SDK and an old version of Milvus, the following message will show using pymilvus

The server does not support Search Iterator V2. Please upgrade your Milvus server, or create a search_iterator with use_v1=True instead

PwzXxm commented 1 hour ago

/run-cpu-e2e

sre-ci-robot commented 1 hour ago

New changes are detected. LGTM label has been removed.

mergify[bot] commented 1 hour ago

@PwzXxm go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify[bot] commented 47 minutes ago

@PwzXxm cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.