milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.87k stars 2.94k forks source link

enhance: Use load pool for `CreateTextIndex` #37898

Closed congqixia closed 13 hours ago

congqixia commented 1 day ago

Related to #37895

Only resolves the starving issue which caused goroutine leakage

codecov[bot] commented 1 day ago

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 81.02%. Comparing base (c07f056) to head (e52eac5). Report is 16 commits behind head on master.

Files with missing lines Patch % Lines
internal/querynodev2/segments/segment.go 0.00% 1 Missing :warning:
Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/graphs/tree.svg?width=650&height=150&src=pr&token=XdCbzGJk2E&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)](https://app.codecov.io/gh/milvus-io/milvus/pull/37898?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) ```diff @@ Coverage Diff @@ ## master #37898 +/- ## ========================================== - Coverage 82.90% 81.02% -1.89% ========================================== Files 1068 1357 +289 Lines 164706 190143 +25437 ========================================== + Hits 136554 154061 +17507 - Misses 22679 30606 +7927 - Partials 5473 5476 +3 ``` | [Components](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | Coverage Δ | | |---|---|---| | [Client](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `72.16% <ø> (ø)` | | | [Core](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `68.87% <ø> (∅)` | | | [Go](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | `83.19% <0.00%> (-0.01%)` | :arrow_down: | | [Files with missing lines](https://app.codecov.io/gh/milvus-io/milvus/pull/37898?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io) | Coverage Δ | | |---|---|---| | [internal/querynodev2/segments/segment.go](https://app.codecov.io/gh/milvus-io/milvus/pull/37898?src=pr&el=tree&filepath=internal%2Fquerynodev2%2Fsegments%2Fsegment.go&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io#diff-aW50ZXJuYWwvcXVlcnlub2RldjIvc2VnbWVudHMvc2VnbWVudC5nbw==) | `66.52% <0.00%> (ø)` | | ... and [312 files with indirect coverage changes](https://app.codecov.io/gh/milvus-io/milvus/pull/37898/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=milvus-io)

🚨 Try these New Features:

mergify[bot] commented 1 day ago

@congqixia E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

congqixia commented 1 day ago

/run-cpu-e2e

xiaofan-luan commented 16 hours ago

Why to use a load pool for index building? That doesn't seems to be a good fix. /hold

congqixia commented 15 hours ago

@xiaofan-luan it was building temporary text index in load process. At least it shall not use dynamic pool. Don't think it's a good idea to open other pool in querynode.

xiaofan-luan commented 15 hours ago

/lgtm /approve

sre-ci-robot commented 15 hours ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: congqixia, xiaofan-luan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[internal/querynodev2/segments/OWNERS](https://github.com/milvus-io/milvus/blob/master/internal/querynodev2/segments/OWNERS)~~ [congqixia,xiaofan-luan] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
congqixia commented 13 hours ago

/unhold