search

pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.25k stars 5.84k forks source link

concurrently processing parquet files will consume a lot of memory, which will cause Lightning OOM [restore for 48576] #56844

Open ti-chi-bot opened 2 weeks ago

ti-chi-bot commented 2 weeks ago

Enhancement

https://pingcap.feishu.cn/wiki/wikcnsWqAhawzcb2SJQ6d4LXRt3 Flow control based on memory limits

ti-chi-bot commented 2 weeks ago

This issue is restored for #48576, it originally created at 2023-11-14 03:31:45.

wuhuizuo commented 2 weeks ago

/cc @BornChanger

wuhuizuo commented 2 weeks ago

/assign @BornChanger