TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
Describe the feature you'd like:
tidb fetches limit+offset number of rows from every partition into memory, then do sort merge, that's why memory consumption is proportional to number of partitions.
tidb could do sort merge on the fly while fetching data from all partitions, so that total memory consumption will not be proportional to number of partitions.
Describe alternatives you've considered:
Teachability, Documentation, Adoption, Migration Strategy:
The above SQLs are from PingCAP paid customer.
Feature Request
Is your feature request related to a problem? Please describe: Memory consumption increase with number of partitions in table:
Describe the feature you'd like: tidb fetches limit+offset number of rows from every partition into memory, then do sort merge, that's why memory consumption is proportional to number of partitions.
tidb could do sort merge on the fly while fetching data from all partitions, so that total memory consumption will not be proportional to number of partitions.
Describe alternatives you've considered:
Teachability, Documentation, Adoption, Migration Strategy: The above SQLs are from PingCAP paid customer.