search

apache / gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
https://gravitino.apache.org
Apache License 2.0
1.1k stars 345 forks source link

[Improvement] The error like "Table ** does not exist in the internal connector" frequently occurs when switch off connector between create data and query data. #5669

Open danhuawang opened 4 hours ago

danhuawang commented 4 hours ago

What would you like to be improved?

When I create and insert data in catalog using gravitino connector ,then query data using cascading connector , I will encounter the following error.

The error like "Table gt_db1.** does not exist in the internal connector" frequently occurs when switch off connector between create data and query data.

2024-11-23 10:26:07 INFO  [main] TrinoQueryIT:93 - Test query env parameters: {postgresql_uri=jdbc:postgresql://10.20.31.18:5432, gravitino_uri=http://10.20.31.21:8090, trino_remote_jdbc_uri=jdbc:trino://10.20.31.22:8080, hive_uri=thrift://10.20.31.20:9083, trino_uri=http://10.20.31.21:8080, mysql_uri=jdbc:mysql://10.20.31.19, hdfs_uri=hdfs://10.20.31.20:9000}
2024-11-23 10:26:07 INFO  [pool-2-thread-1] TrinoQueryIT:188 - Execute sql in the tester jdbc-mysql/catalog_mysql_prepare.sql under catalog mysql :
call gravitino.system.create_catalog(
    'gt_mysql1',
    'jdbc-mysql',
    map(
        array['jdbc-url', 'jdbc-user', 'jdbc-password', 'jdbc-driver', 'trino.bypass.join-pushdown.strategy', 'cloud.region-code', 'cloud.trino.connection-url', 'cloud.trino.connection-user', 'cloud.trino.connection-password'],
        array['jdbc:mysql://10.20.31.19/?useSSL=false', 'trino', 'ds123', 'com.mysql.cj.jdbc.Driver', 'EAGER','c2', 'jdbc:trino://10.20.31.22:8080', 'admin', '']
    )
)
Result:
CALL

2024-11-23 10:26:07 INFO  [pool-2-thread-1] TrinoQueryIT:188 - Execute sql in the tester jdbc-mysql/catalog_mysql_prepare.sql under catalog mysql :
call gravitino.system.create_catalog(
    'gt_mysql1_1',
    'jdbc-mysql',
    map(
        array['jdbc-url', 'jdbc-user', 'jdbc-password', 'jdbc-driver', 'trino.bypass.join-pushdown.strategy'],
        array['jdbc:mysql://10.20.31.19/?useSSL=false', 'trino', 'ds123', 'com.mysql.cj.jdbc.Driver', 'EAGER']
    )
)
Result:
CALL

2024-11-23 10:26:08 INFO  [pool-2-thread-1] TrinoQueryIT:266 - Execute sql in the tester jdbc-mysql/00001_datatype.sql under catalog mysql successfully.
Sql:
CREATE SCHEMA gt_mysql1_1.gt_db1;
Expect:
CREATE SCHEMA
Actual:
CREATE SCHEMA
2024-11-23 10:26:08 INFO  [pool-2-thread-1] TrinoQueryIT:266 - Execute sql in the tester jdbc-mysql/00001_datatype.sql under catalog mysql successfully.
Sql:
CREATE TABLE gt_mysql1_1.gt_db1.tb03 (id int, name char(20));
Expect:
CREATE TABLE
Actual:
CREATE TABLE
2024-11-23 10:26:08 ERROR [pool-2-thread-1] TrinoQueryIT:279 - Execute sql in the tester jdbc-mysql/00001_datatype.sql under catalog mysql failed.
Sql:
SELECT * FROM gt_mysql1.gt_db1.tb03;
Expect:
<BLANK_LINE>
Actual:
Query 20241123_022525_00151_b88tc failed: Table gt_db1.tb03 does not exist in the internal connector
io.trino.spi.TrinoException: Table gt_db1.tb03 does not exist in the internal connector
        at org.apache.gravitino.trino.connector.GravitinoMetadata.getTableHandle(GravitinoMetadata.java:120)
        at org.apache.gravitino.trino.connector.GravitinoMetadata.getTableHandle(GravitinoMetadata.java:75)
        at io.trino.tracing.TracingConnectorMetadata.getTableHandle(TracingConnectorMetadata.java:149)
        at io.trino.metadata.MetadataManager.lambda$getTableHandle$5(MetadataManager.java:285)
        at java.base/java.util.Optional.flatMap(Optional.java:289)
        at io.trino.metadata.MetadataManager.getTableHandle(MetadataManager.java:279)
        at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1848)
        at io.trino.metadata.MetadataManager.getRedirectionAwareTableHandle(MetadataManager.java:1840)
        at io.trino.tracing.TracingMetadata.getRedirectionAwareTableHandle(TracingMetadata.java:1423)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.getTableHandle(StatementAnalyzer.java:5816)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:2279)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitTable(StatementAnalyzer.java:521)
        at io.trino.sql.tree.Table.accept(Table.java:60)
        at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:540)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.analyzeFrom(StatementAnalyzer.java:4869)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:3062)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuerySpecification(StatementAnalyzer.java:521)
        at io.trino.sql.tree.QuerySpecification.accept(QuerySpecification.java:155)
        at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:540)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:548)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:1558)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.visitQuery(StatementAnalyzer.java:521)
        at io.trino.sql.tree.Query.accept(Query.java:118)
        at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
        at io.trino.sql.analyzer.StatementAnalyzer$Visitor.process(StatementAnalyzer.java:540)
        at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:500)
        at io.trino.sql.analyzer.StatementAnalyzer.analyze(StatementAnalyzer.java:489)
        at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:97)
        at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:86)
        at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:274)
        at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:209)
        at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:850)
        at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:153)
        at io.trino.$gen.Trino_435____20241123_020935_2.call(Unknown Source)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:76)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

How should we improve?

No response

danhuawang commented 4 hours ago

@diqiu50 Can you help resolve this problem ?