search

pingcap / tiflash

The analytical engine for TiDB and TiDB Cloud. Try free: https://tidbcloud.com/free-trial
https://docs.pingcap.com/tidb/stable/tiflash-overview
Apache License 2.0
948 stars 410 forks source link

tpch 500G get some error #6051

Open RenlySir opened 2 years ago

RenlySir commented 2 years ago

General Question

RenlySir commented 2 years ago

i do tpch 500g,when i set tidb_allow_mpp=1 and set tidb_isolation_read_engines =['tidb','tiflash']. run Q3,i get error like " ERROR 1105 (HY000): other error for mpp stream: Code: 0, e.displayText() = DB::Exception: Exchange receiver meet error : Code: 0, e.displayText() = DB::Exception: Memory limit (for query) exceeded: would use 93.14 GiB (attempt to allocate chunk of 41943040 bytes), maximum: 93.13 GiB, e.what() = DB::Exception,, e.what() = DB::Exception," the memory where tiflash deploy machine is 256G. tiflash mpp get too much memory. this is not good. i want to know the plan about tiflash mpp

JinheLin commented 2 years ago

Please check the following configurations.

[profiles.default]
    ## The memory usage limit for the generated intermediate data when a single
    ## coprocessor query is executed. The default value is 0, which means no limit.
    max_memory_usage = 0

    ## The memory usage limit for the generated intermediate data when all queries
    ## are executed. The default value is 0 (in bytes), which means no limit.
    max_memory_usage_for_all_queries = 0

https://docs.pingcap.com/tidb/dev/tiflash-configuration

RenlySir commented 2 years ago

Please check the following configurations.

[profiles.default]
    ## The memory usage limit for the generated intermediate data when a single
    ## coprocessor query is executed. The default value is 0, which means no limit.
    max_memory_usage = 0

    ## The memory usage limit for the generated intermediate data when all queries
    ## are executed. The default value is 0 (in bytes), which means no limit.
    max_memory_usage_for_all_queries = 0

https://docs.pingcap.com/tidb/dev/tiflash-configuration

conf is as this: tiflash: logger.count: 5 logger.level: warning profiles.default.batch_cop_pool_size: 32 profiles.default.cop_pool_size: 32 profiles.default.max_memory_usage: 0 profiles.default.max_memory_usage_for_all_queries: 0