Open Longxiang-Bai opened 1 year ago
Getting this error on extractTableNamesFromScript is very unexpected, since it doesn't actually serialize the AST. I would expect this on almost any other API call, are you sure it's that call that is failing?
I assume the raw sql itself doesn't acceed this size?
Hi @matthewcbrown, thanks for your reply. I am pretty sure it is this API. Since our sql has over 400k parameters lol. So it is possible it actually exceeds the size. Do you have any comment or suggestions?
I experienced this error as well. However, the circumstances which it arises is different.
We are only running a simple SELECT 1
statement with ZetaSQL Toolkit, but with minor difference with regards how the catalog is build with.
If we build the catalog with a project with a lot of tables (415 tables) (:
catalog.addAllTablesInProject("projectId")
(note that this is ZetaSQL Toolkit API and what it does is to populate the SimpleCatalog
)
We will encounter similar error:
Exception in thread "main" com.google.zetasql.toolkit.AnalysisException: Received message larger than max (4266687 vs. 4194304)
at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.analyzeNextStatement(ZetaSQLToolkitAnalyzer.java:232)
at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.next(ZetaSQLToolkitAnalyzer.java:211)
at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.next(ZetaSQLToolkitAnalyzer.java:148)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at com.google.zetasql.toolkit.examples.AnalyzeBigQuery.main(AnalyzeBigQuery.java:79)
Caused by: com.google.zetasql.SqlException: Received message larger than max (4266687 vs. 4194304)
at com.google.zetasql.Analyzer.analyzeNextStatement(Analyzer.java:220)
at com.google.zetasql.toolkit.ZetaSQLToolkitAnalyzer$StatementAnalyzer.analyzeNextStatement(ZetaSQLToolkitAnalyzer.java:229)
... 4 more
Caused by: com.google.zetasql.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (4266687 vs. 4194304)
at com.google.zetasql.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)
at com.google.zetasql.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)
at com.google.zetasql.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)
at com.google.zetasql.ZetaSqlLocalServiceGrpc$ZetaSqlLocalServiceBlockingStub.analyze(ZetaSqlLocalServiceGrpc.java:1506)
at com.google.zetasql.Analyzer.analyzeNextStatement(Analyzer.java:218)
... 5 more
If the catalog has small number of tables (45 in our case), this error will not occur.
Hi, we are using zetasql 2023.04.1 and we are facing the following error:
com.google.zetasql.io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: Received message larger than max (4224741 vs. 4194304)
when processing a huge sql withAnalyzer.extractTableNamesFromScript()
. I am wondering if anyone could advise how can we make it work? Thanks!