search

pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
948 stars 410 forks source link

The mpp query raise TiFlash oom in the CH-benCHmark workload #1995

Open lilinghai opened 3 years ago

lilinghai commented 3 years ago

CH-benCHmark 10k warehouses data and tiflash replica is 2 in master cluster Execute the ap query select count(*) from revenue1; with one thread, The Tiflash memory raise about 130GB. If execute the sql with 2 threads, The Tiflash crash for oom. image

zanmato1984 commented 3 years ago

revenue1 is a view that expands to a big join + agg. Total dataset sizes about 750GB. It is expected to have a large memory footprint for such a query.

We plan to have memory control but that will wait for 1 or 2 sprints.

Lower the priority to major.

zanmato1984 commented 2 years ago

Changing type to enhancement.