search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
36.8k stars 5.8k forks source link

Executor: dynamically changing the concurrency of the Scan operator #18448

Open fzhedu opened 4 years ago

fzhedu commented 4 years ago

Feature Request

Is your feature request related to a problem? Please describe:

https://github.com/pingcap/tidb/issues/18444, after increasing the concurrency of operators but decreasing the concurrency of Scan, the OOM disappears.

Describe the feature you'd like:

dynamically channging the concurrency of the scan operator, because the default 15 concurrency would buffer so many regions that occupy a lot of memory. This feature would avoid OOM in many cases.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

SunRunAway commented 4 years ago

Does https://github.com/pingcap/tidb/pull/17324 fix this issue?

fzhedu commented 4 years ago

Does #17324 fix this issue?

no. I planned let the PR fix this issue, but let it as the TODO work. It should not open too many copWorkers at the beginning.