search

tidb-incubator / TiBigData

TiDB connectors for Flink/Hive/Presto
Apache License 2.0
211 stars 57 forks source link

Unable to start Trino after using PrestoSQL-TiDB-Connector #110

Open myheartsgoon opened 2 years ago

myheartsgoon commented 2 years ago

I followed this doc to build the connector and the build finished successfully. https://github.com/tidb-incubator/TiBigData/tree/master/prestosql

my tidb plugin folder content as blew:

root@trino-worker01:/home/trino/trino-server-361/etc/catalog# ls -l /home/trino/trino-server-361/plugin/tidb/
total 38940
-rw-r--r-- 1 root root     4467 9月  28 15:03 aopalliance-1.0.jar
-rw-r--r-- 1 root root    44329 9月  28 15:03 bigdata-core-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root    26353 9月  28 15:03 bootstrap-202.jar
-rw-r--r-- 1 root root   805301 9月  28 15:03 bval-jsr-2.0.0.jar
-rw-r--r-- 1 root root   415348 9月  28 15:03 cglib-nodep-3.3.0.jar
-rw-r--r-- 1 root root   230905 9月  28 15:03 checker-qual-3.8.0.jar
-rw-r--r-- 1 root root    83762 9月  28 15:03 configuration-202.jar
-rw-r--r-- 1 root root    13854 9月  28 15:03 error_prone_annotations-2.5.1.jar
-rw-r--r-- 1 root root     4617 9月  28 15:03 failureaccess-1.0.1.jar
-rw-r--r-- 1 root root  2874025 9月  28 15:03 guava-30.1.1-jre.jar
-rw-r--r-- 1 root root   856934 9月  28 15:03 guice-4.2.3.jar
-rw-r--r-- 1 root root   156005 9月  28 15:03 HikariCP-3.4.5.jar
-rw-r--r-- 1 root root     8781 9月  28 15:03 j2objc-annotations-1.3.jar
-rw-r--r-- 1 root root    68167 9月  28 15:03 jackson-annotations-2.11.1.jar
-rw-r--r-- 1 root root   349108 9月  28 15:03 jackson-core-2.10.3.jar
-rw-r--r-- 1 root root  1404171 9月  28 15:03 jackson-databind-2.10.3.jar
-rw-r--r-- 1 root root    83893 9月  28 15:03 jackson-datatype-guava-2.10.3.jar
-rw-r--r-- 1 root root    34402 9月  28 15:03 jackson-datatype-jdk8-2.10.3.jar
-rw-r--r-- 1 root root    72117 9月  28 15:03 jackson-datatype-joda-2.10.3.jar
-rw-r--r-- 1 root root   105898 9月  28 15:03 jackson-datatype-jsr310-2.10.3.jar
-rw-r--r-- 1 root root     9328 9月  28 15:03 jackson-module-parameter-names-2.10.3.jar
-rw-r--r-- 1 root root    26586 9月  28 15:03 javax.annotation-api-1.3.2.jar
-rw-r--r-- 1 root root     2497 9月  28 15:03 javax.inject-1.jar
-rw-r--r-- 1 root root    16537 9月  28 15:03 jcl-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root  2920901 9月  28 15:03 jmxutils-1.21.jar
-rw-r--r-- 1 root root    32601 9月  28 15:03 json-202.jar
-rw-r--r-- 1 root root    19936 9月  28 15:03 jsr305-3.0.2.jar
-rw-r--r-- 1 root root     2199 9月  28 15:03 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
-rw-r--r-- 1 root root     3908 9月  28 15:03 log-202.jar
-rw-r--r-- 1 root root    23702 9月  28 15:03 log4j-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root   471901 9月  28 15:03 logback-core-1.2.3.jar
-rw-r--r-- 1 root root    19470 9月  28 15:03 log-manager-202.jar
-rw-r--r-- 1 root root  1006956 9月  28 15:45 mysql-connector-java-5.1.48.jar
-rw-r--r-- 1 root root    48203 9月  28 15:03 prestosql-connector-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root    41203 9月  28 15:03 slf4j-api-1.7.25.jar
-rw-r--r-- 1 root root     8510 9月  28 15:03 slf4j-jdk14-1.7.30.jar
-rw-r--r-- 1 root root   103273 9月  28 15:03 tidb-jdbc-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root 27276468 9月  28 15:03 tikv-client-java-3.2.0-SNAPSHOT.jar
-rw-r--r-- 1 root root    19555 9月  28 15:03 units-1.6.jar
-rw-r--r-- 1 root root    93107 9月  28 15:03 validation-api-2.0.1.Final.jar

And I add the catalog for tidb, content as blow.

connector.name=tidb
tidb.database.url=jdbc:mysql://xxx:3306/xxx
tidb.username=xxx
tidb.password=xxx
tidb.filter-push-down=true

But after I restart the trino, it errored as below.

2021-09-28T15:49:51.601+0800    ERROR   main    io.trino.server.Server  No service providers of type io.trino.spi.Plugin
java.lang.IllegalStateException: No service providers of type io.trino.spi.Plugin
        at com.google.common.base.Preconditions.checkState(Preconditions.java:591)
        at io.trino.server.PluginManager.loadPlugin(PluginManager.java:139)
        at io.trino.server.PluginManager.loadPlugin(PluginManager.java:129)
        at io.trino.server.ServerPluginsProvider.loadPlugins(ServerPluginsProvider.java:48)
        at io.trino.server.PluginManager.loadPlugins(PluginManager.java:110)
        at io.trino.server.Server.doStart(Server.java:122)
        at io.trino.server.Server.lambda$start$0(Server.java:77)
        at io.trino.$gen.Trino_361____20210928_074943_1.run(Unknown Source)
        at io.trino.server.Server.start(Server.java:77)
        at io.trino.server.TrinoServer.main(TrinoServer.java:38)

My trino version: 361 mysql connector I use: mysql-connector-java-5.1.48.jar

Please help if anything wrong with my setup?

myheartsgoon commented 2 years ago

I manage to fixe the above issue by change prestosql to trino in build process. Now I am encountering below issue when start the trino server.

error when adding tidb.filter-push-down=true in the catalog file

If I add tidb.filter-push-down=true in the catalog file, below error will occurr.

2021-09-28T17:11:03.612+0800    ERROR   main    io.trino.server.Server  Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
    at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
    at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
    at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
    at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
    at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
    at io.trino.server.Server.doStart(Server.java:124)
    at io.trino.server.Server.lambda$start$0(Server.java:77)
    at io.trino.$gen.Trino_361____20210928_091053_1.run(Unknown Source)
    at io.trino.server.Server.start(Server.java:77)
    at io.trino.server.TrinoServer.main(TrinoServer.java:38)

2021-09-28T17:11:03.614+0800    INFO    Thread-81   io.airlift.bootstrap.LifeCycleManager   JVM is shutting down, cleaning up
2021-09-28T17:11:03.614+0800    INFO    Thread-81   io.airlift.bootstrap.LifeCycleManager   Life cycle stopping...
2021-09-28T17:11:03.615+0800    INFO    Thread-81   io.airlift.bootstrap.LifeCycleManager   Life cycle stopped

error WITHOUT adding tidb.filter-push-down=true in the catalog file

If remove tidb.filter-push-down=true, then server can be started successfully, but when I run query in cli, Internal error will occurred.

trino:skyline_test> select * from test1;
Query 20210928_101552_00008_hcyiy failed: Internal error

In the stack trace log it shows as below:

java.lang.UnsupportedOperationException: undefined
    at io.trino.spi.connector.ConnectorSplitManager.getSplits(ConnectorSplitManager.java:35)
    at io.trino.split.SplitManager.getSplits(SplitManager.java:86)
    at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitScanAndFilter(DistributedExecutionPlanner.java:206)
    at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:188)
    at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:159)
    at io.trino.sql.planner.plan.TableScanNode.accept(TableScanNode.java:222)
    at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:127)
    at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:134)
    at io.trino.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:104)
    at io.trino.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:508)
    at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:415)
    at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:237)
    at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
    at io.trino.$gen.Trino_361____20210928_100845_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
sunxiaoguang commented 2 years ago

tidb.filter-push-down

Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?

myheartsgoon commented 2 years ago

tidb.filter-push-down

Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?

Thanks for your prompt response, I just tested with version 359, when not specifying tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)

When I adding tidb.filter-push-down=true in catalog file, it is still not working, same error message as I describe above. 

2021-09-29T11:47:43.085+0800    ERROR   main    io.trino.server.Server  Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:

1) Error: Configuration property 'tidb.filter-push-down' was not used

1 error
    at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
    at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
    at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
    at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
    at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
    at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
    at io.trino.server.Server.doStart(Server.java:124)
    at io.trino.server.Server.lambda$start$0(Server.java:77)
    at io.trino.$gen.Trino_359____20210929_034733_1.run(Unknown Source)
    at io.trino.server.Server.start(Server.java:77)
    at io.trino.server.TrinoServer.main(TrinoServer.java:38)

But for now I am good as at least I can query tidb data with presto, although without filter pushdown feature.

sunxiaoguang commented 2 years ago

Thanks for your prompt response, I just tested with version 359, when not specifying tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)

Trino / PrestoSQL / PrestoDB are evolving pretty fast, it's usually required to match version used to compile and run connector.

@humengyu2012 Can you take a look at configuration not used error?

humengyu2012 commented 2 years ago

Presto will always enable filter-push-down and can not be disabled, so you don't need to configure it.

humengyu2012 commented 2 years ago

This is our fault, we will change the documentation later.