0x822a5b87 / test-calcite

test calcite
Apache License 2.0
0 stars 0 forks source link

请教使用calcite创建视图问题 #1

Open AKang242411 opened 2 months ago

AKang242411 commented 2 months ago

你好,通过修改您连接mysql的例子,如下: { "version": "1.0", "defaultSchema": "MYSQL", "schemas": [ { "name": "MYSQL", "type": "custom", "factory": "com.xxx.calcite.mysql.MyMysqlSchemaFactory", "operand": { "url": "jdbc:mysql://localhost:3306/iam_service?serverTimezone=UTC", "user": "root", "pass": "***" }, "functions": [ { "name": "test", "className": "com.xxx.calcite.function.MyFunction", "methodName": "test" } ], "tables": [ { "name": "v_num", "type": "view", "sql": "select from \"iam_sys_user\"", "path": [ "MYSQL" ], "modifiable": false } ] } ] } @Test void validateView() throws SQLException, SqlParseException { String sql = "select \"user_name\" from MYSQL.\"v_num\""; executeQuery(sql); } 执行上面测试,不成功 报错信息: java.sql.SQLException: Error while executing SQL "select "user_name" from MYSQL."v_num"": There are not enough rules to produce a node with desired properties: convention=ENUMERABLE, sort=[]. Missing conversion is LogicalTableScan[convention: NONE -> ENUMERABLE] There is 1 empty subset: rel#24:RelSubset#0.ENUMERABLE.[], the relevant part of the original plan is as follows 8:LogicalTableScan(table=[[MYSQL, iam_sys_user]])

Root: rel#22:RelSubset#1.ENUMERABLE.[] Original rel: LogicalProject(user_name=[$2]): rowcount = 100.0, cumulative cost = {200.0 rows, 201.0 cpu, 0.0 io}, id = 17 LogicalTableScan(table=[[MYSQL, iam_sys_user]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 8

Sets: Set#0, type: RecordType(BIGINT user_id, BIGINT dept_id, VARCHAR user_name, VARCHAR nick_name, VARCHAR user_type, VARCHAR email, VARCHAR id_card, VARCHAR phonenumber, CHAR(1) sex, VARCHAR avatar, VARCHAR password, CHAR(1) status, CHAR(1) del_flag, VARCHAR login_ip, TIMESTAMP(0) login_date, VARCHAR create_by, TIMESTAMP(0) create_time, VARCHAR update_by, TIMESTAMP(0) update_time, VARCHAR remark, VARCHAR signature, TIMESTAMP(0) password_reset_time) rel#18:RelSubset#0.NONE.[], best=null rel#8:LogicalTableScan.NONE.[](table=[MYSQL, iam_sys_user]), rowcount=100.0, cumulative cost={inf} rel#24:RelSubset#0.ENUMERABLE.[], best=null Set#1, type: RecordType(VARCHAR user_name) rel#20:RelSubset#1.NONE.[], best=null rel#19:LogicalProject.NONE., rowcount=100.0, cumulative cost={inf} rel#22:RelSubset#1.ENUMERABLE.[], best=null rel#23:AbstractConverter.ENUMERABLE., rowcount=100.0, cumulative cost={inf} rel#25:EnumerableProject.ENUMERABLE., rowcount=100.0, cumulative cost={inf}

Graphviz: digraph G { root [style=filled,label="Root"]; subgraph cluster0{ label="Set 0 RecordType(BIGINT user_id, BIGINT dept_id, VARCHAR user_name, VARCHAR nick_name, VARCHAR user_type, VARCHAR email, VARCHAR id_card, VARCHAR phonenumber, CHAR(1) sex, VARCHAR avatar, VARCHAR password, CHAR(1) status, CHAR(1) del_flag, VARCHAR login_ip, TIMESTAMP(0) login_date, VARCHAR create_by, TIMESTAMP(0) create_time, VARCHAR update_by, TIMESTAMP(0) update_time, VARCHAR remark, VARCHAR signature, TIMESTAMP(0) password_reset_time)"; rel8 [label="rel#8:LogicalTableScan\ntable=[MYSQL, iam_sys_user]\nrows=100.0, cost={inf}",shape=box] subset18 [label="rel#18:RelSubset#0.NONE.[]"] subset24 [label="rel#24:RelSubset#0.ENUMERABLE.[]",color=red] } subgraph cluster1{ label="Set 1 RecordType(VARCHAR user_name)"; rel19 [label="rel#19:LogicalProject\ninput=RelSubset#18,exprs=[$2]\nrows=100.0, cost={inf}",shape=box] rel23 [label="rel#23:AbstractConverter\ninput=RelSubset#20,convention=ENUMERABLE,sort=[]\nrows=100.0, cost={inf}",shape=box] rel25 [label="rel#25:EnumerableProject\ninput=RelSubset#24,exprs=[$2]\nrows=100.0, cost={inf}",shape=box] subset20 [label="rel#20:RelSubset#1.NONE.[]"] subset22 [label="rel#22:RelSubset#1.ENUMERABLE.[]"] } root -> subset22; subset18 -> rel8; subset20 -> rel19; rel19 -> subset18; subset22 -> rel23; rel23 -> subset20; subset22 -> rel25; rel25 -> subset24; }

at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at com.xxx.calcite.ModelTest.executeQuery(ModelTest.java:55)
at com.xxx.calcite.ModelTest.validateView(ModelTest.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

Caused by: org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not enough rules to produce a node with desired properties: convention=ENUMERABLE, sort=[]. Missing conversion is LogicalTableScan[convention: NONE -> ENUMERABLE] There is 1 empty subset: rel#24:RelSubset#0.ENUMERABLE.[], the relevant part of the original plan is as follows 8:LogicalTableScan(table=[[MYSQL, iam_sys_user]])

Root: rel#22:RelSubset#1.ENUMERABLE.[] Original rel: LogicalProject(user_name=[$2]): rowcount = 100.0, cumulative cost = {200.0 rows, 201.0 cpu, 0.0 io}, id = 17 LogicalTableScan(table=[[MYSQL, iam_sys_user]]): rowcount = 100.0, cumulative cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 8

Sets: Set#0, type: RecordType(BIGINT user_id, BIGINT dept_id, VARCHAR user_name, VARCHAR nick_name, VARCHAR user_type, VARCHAR email, VARCHAR id_card, VARCHAR phonenumber, CHAR(1) sex, VARCHAR avatar, VARCHAR password, CHAR(1) status, CHAR(1) del_flag, VARCHAR login_ip, TIMESTAMP(0) login_date, VARCHAR create_by, TIMESTAMP(0) create_time, VARCHAR update_by, TIMESTAMP(0) update_time, VARCHAR remark, VARCHAR signature, TIMESTAMP(0) password_reset_time) rel#18:RelSubset#0.NONE.[], best=null rel#8:LogicalTableScan.NONE.[](table=[MYSQL, iam_sys_user]), rowcount=100.0, cumulative cost={inf} rel#24:RelSubset#0.ENUMERABLE.[], best=null Set#1, type: RecordType(VARCHAR user_name) rel#20:RelSubset#1.NONE.[], best=null rel#19:LogicalProject.NONE., rowcount=100.0, cumulative cost={inf} rel#22:RelSubset#1.ENUMERABLE.[], best=null rel#23:AbstractConverter.ENUMERABLE., rowcount=100.0, cumulative cost={inf} rel#25:EnumerableProject.ENUMERABLE., rowcount=100.0, cumulative cost={inf}

Graphviz: digraph G { root [style=filled,label="Root"]; subgraph cluster0{ label="Set 0 RecordType(BIGINT user_id, BIGINT dept_id, VARCHAR user_name, VARCHAR nick_name, VARCHAR user_type, VARCHAR email, VARCHAR id_card, VARCHAR phonenumber, CHAR(1) sex, VARCHAR avatar, VARCHAR password, CHAR(1) status, CHAR(1) del_flag, VARCHAR login_ip, TIMESTAMP(0) login_date, VARCHAR create_by, TIMESTAMP(0) create_time, VARCHAR update_by, TIMESTAMP(0) update_time, VARCHAR remark, VARCHAR signature, TIMESTAMP(0) password_reset_time)"; rel8 [label="rel#8:LogicalTableScan\ntable=[MYSQL, iam_sys_user]\nrows=100.0, cost={inf}",shape=box] subset18 [label="rel#18:RelSubset#0.NONE.[]"] subset24 [label="rel#24:RelSubset#0.ENUMERABLE.[]",color=red] } subgraph cluster1{ label="Set 1 RecordType(VARCHAR username)"; rel19 [label="rel#19:LogicalProject\ninput=RelSubset#18,exprs=[$2]\nrows=100.0, cost={inf}",shape=box] rel23 [label="rel#23:AbstractConverter\ninput=RelSubset#20,convention=ENUMERABLE,sort=[]\nrows=100.0, cost={inf}",shape=box] rel25 [label="rel#25:EnumerableProject\ninput=RelSubset#24,exprs=[$2]\nrows=100.0, cost={inf}",shape=box] subset20 [label="rel#20:RelSubset#1.NONE.[]"] subset22 [label="rel#22:RelSubset#1.ENUMERABLE.[]"] } root -> subset22; subset18 -> rel8; subset20 -> rel19; rel19 -> subset18; subset22 -> rel23; rel23 -> subset20; subset22 -> rel25; rel25 -> subset24; } at org.apache.calcite.plan.volcano.RelSubset$CheapestPlanReplacer.visit(RelSubset.java:742) at org.apache.calcite.plan.volcano.RelSubset.buildCheapestPlan(RelSubset.java:365) at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:520) at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:271) at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:331) at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:166) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:208) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2(CalcitePrepareImpl.java:642) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:508) at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:478) at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231) at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:556) at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156) ... 70 more

0x822a5b87 commented 2 months ago

我这边没办法复现你的异常,但是看这段报错:

Caused by: org.apache.calcite.plan.RelOptPlanner$CannotPlanException: There are not enough rules to produce a node with desired properties: convention=ENUMERABLE, sort=[].

这个应该是发生在优化阶段,将一个RelNode转化为一个等价的RelNode以提升查询性能。

另外,在异常的最后一部分:

at org.apache.calcite.plan.volcano.RelSubset$CheapestPlanReplacer.visit(RelSubset.java:742)
at org.apache.calcite.plan.volcano.RelSubset.buildCheapestPlan(RelSubset.java:365)
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:520)
at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:271)

这里的异常应该是在 findBestExp 这里,你可以试试单步调试一下这个位置,看看具体引起异常的原因是什么。

我这边用的最新版本的 1.37.0 下,代码应该是在这个 RelSubset#CheapestPlanReplacer#visit() 方法抛出的异常:

    public RelNode visit(
        RelNode p,
        int ordinal,
        @Nullable RelNode parent) {
          // 优化阶段
          planner.dump(pw);
          pw.flush();
          final String dump = sw.toString();
          RuntimeException e =
              new RelOptPlanner.CannotPlanException(dump);
          LOGGER.trace("Caught exception in class={}, method=visit", getClass().getName(), e);
          throw e;
}
AKang242411 commented 2 months ago

你好,请教一下,calcite能够实现上述功能么,连接数据库,创建一个视图,再通过视图进行sql查询。查阅了很多资料,一直没有找到这方面的资料,您的calcite-schema-demo,只能创建select 1+2*3这种视图,才能实现。