search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.55k stars 3.03k forks source link

SELECT DISTINCT throws statement is too large (stack overflow during analysis) with a table of +600 columns #12897

Open aletc1 opened 2 years ago

aletc1 commented 2 years ago

The problem is when DISTINCT appears in the query:

SELECT DISTINCT col1, col2, ..., col600 FROM sometable

The same query without DISTINCT works as expected so the message "statement is too large" is misleading. It is not related to a specific connector because I tested it with different ones.

Stack trace:

io.trino.spi.TrinoException: statement is too large (stack overflow during analysis)
    at io.trino.execution.SqlQueryExecution.planQuery(SqlQueryExecution.java:461)
    at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:399)
    at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:243)
    at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
    at io.trino.$gen.Trino_386____20220618_095239_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.StackOverflowError: undefined
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:471)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:70)
    at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.tree.ExpressionTreeRewriter.rewrite(ExpressionTreeRewriter.java:58)
    at io.trino.sql.tree.ExpressionTreeRewriter.rewrite(ExpressionTreeRewriter.java:50)
    at io.trino.sql.tree.ExpressionTreeRewriter.access$300(ExpressionTreeRewriter.java:25)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:493)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:70)
    at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.tree.ExpressionTreeRewriter.defaultRewrite(ExpressionTreeRewriter.java:67)
    at io.trino.sql.planner.planprinter.PlanPrinter$1.rewriteFunctionCall(PlanPrinter.java:1684)
    at io.trino.sql.planner.planprinter.PlanPrinter$1.rewriteFunctionCall(PlanPrinter.java:1680)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:472)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:70)
    at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.tree.ExpressionTreeRewriter.rewrite(ExpressionTreeRewriter.java:58)
    at io.trino.sql.tree.ExpressionTreeRewriter.rewrite(ExpressionTreeRewriter.java:50)
    at io.trino.sql.tree.ExpressionTreeRewriter.access$300(ExpressionTreeRewriter.java:25)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:493)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:70)
    at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
    at io.trino.sql.tree.ExpressionTreeRewriter.defaultRewrite(ExpressionTreeRewriter.java:67)
    at io.trino.sql.planner.planprinter.PlanPrinter$1.rewriteFunctionCall(PlanPrinter.java:1684)
    at io.trino.sql.planner.planprinter.PlanPrinter$1.rewriteFunctionCall(PlanPrinter.java:1680)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:472)
    at io.trino.sql.tree.ExpressionTreeRewriter$RewritingVisitor.visitFunctionCall(ExpressionTreeRewriter.java:70)
    at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
    at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
...
wwbbcjeyc commented 3 months ago

Set this up and try it. set session pre_aggregate_case_aggregations_enabled=false;