search

DataSQRL / sqrl

Flexible development framework for building streaming data applications in SQL with Kafka, Flink, Postgres, GraphQL, and more.
https://www.datasqrl.com/
97 stars 14 forks source link

Exec hint for 'database' resolves to two database engines in test goal (postgres and duckdb) #860

Open mbroecheler opened 1 month ago

mbroecheler commented 1 month ago

Running test with a hint like /*+test,EXEC(database) */ will result in too many database engines being selected. A workaround is to use /*+test,EXEC(postgres) */.

java.lang.IllegalArgumentException: Stream contains [2] elements
        at com.datasqrl.util.StreamUtil.getOnlyElement(StreamUtil.java:29)
        at com.datasqrl.engine.pipeline.ExecutionPipeline.getStageByType(ExecutionPipeline.java:35)
        at com.datasqrl.config.SqrlConfigPipeline.getStageByType(SqrlConfigPipeline.java:12)
        at com.datasqrl.plan.table.QueryRelationalTable.lambda$getSupportedStages$0(QueryRelationalTable.java:53)
        at java.base/java.util.Optional.or(Optional.java:318)
        at com.datasqrl.plan.table.QueryRelationalTable.lambda$getSupportedStages$1(QueryRelationalTable.java:53)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at com.datasqrl.plan.table.QueryRelationalTable.getSupportedStages(QueryRelationalTable.java:57)
        at com.datasqrl.plan.global.DAGBuilder.getInputTable(DAGBuilder.java:101)
        at com.datasqrl.plan.global.DAGBuilder.lambda$add2DAG$4(DAGBuilder.java:90)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
        at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
        at com.datasqrl.plan.global.DAGBuilder.add2DAG(DAGBuilder.java:90)
        at com.datasqrl.plan.global.DAGBuilder.build(DAGBuilder.java:59)
        at com.datasqrl.plan.global.DAGPlanner.build(DAGPlanner.java:53)
        at com.datasqrl.plan.global.DAGPlanner.planLogical(DAGPlanner.java:100)
        at com.datasqrl.compile.CompilationProcess.executeCompilation(CompilationProcess.java:64)
        at com.datasqrl.cmd.AbstractCompilerCommand.execute(AbstractCompilerCommand.java:110)
        at com.datasqrl.cmd.TestCommand.execute(TestCommand.java:32)
        at com.datasqrl.cmd.AbstractCommand.run(AbstractCommand.java:26)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2030)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2465)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2457)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2419)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2421)
        at picocli.CommandLine.execute(CommandLine.java:2174)
        at com.datasqrl.DatasqrlCMD.main(DatasqrlCMD.java:13)
[FATAL] Stream contains [2] elements
henneberger commented 1 month ago

cc @mbroecheler. The dag planner should probably determine the best database engine during planning.