search

UBOdin / mimir

Data-ish exploration through SQL+Uncertainty
http://mimirdb.info
Apache License 2.0
26 stars 13 forks source link

row_number is incorrectly pulled into lazy_row #356

Open okennedy opened 4 years ago

okennedy commented 4 years ago

Should be evaluated by spark.

mimir> select row_number(), cast(h as float)*60*60+cast(m as float)*60+cast(s as float)-75359.239 from extract_warm_start;
java.lang.RuntimeException: Error Decoding ROW_NUMBER (int)
        at mimir.exec.result.LazyRow.apply(LazyRow.scala:22)
        at mimir.exec.PrettyOutputFormat$$anonfun$print$4$$anonfun$apply$1.apply$mcVI$sp(OutputFormat.scala:81)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
        at mimir.exec.PrettyOutputFormat$$anonfun$print$4.apply(OutputFormat.scala:80)
        at mimir.exec.PrettyOutputFormat$$anonfun$print$4.apply(OutputFormat.scala:79)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at mimir.exec.result.ResultSeq.foreach(ResultSeq.scala:5)
        at mimir.exec.PrettyOutputFormat.print(OutputFormat.scala:79)
        at mimir.Mimir$$anonfun$handleQuery$1$$anonfun$apply$mcV$sp$1.apply(Mimir.scala:204)
        at mimir.Mimir$$anonfun$handleQuery$1$$anonfun$apply$mcV$sp$1.apply(Mimir.scala:204)
        at mimir.Database.query(Database.scala:155)
        at mimir.Database.query(Database.scala:182)
        at mimir.Mimir$$anonfun$handleQuery$1.apply$mcV$sp(Mimir.scala:204)
        at mimir.Mimir$$anonfun$handleQuery$1.apply(Mimir.scala:204)
        at mimir.Mimir$$anonfun$handleQuery$1.apply(Mimir.scala:204)
        at mimir.util.Timer$.monitor(Timer.scala:22)
        at mimir.Mimir$.handleQuery(Mimir.scala:203)
        at mimir.Mimir$.handleSelect(Mimir.scala:198)
        at mimir.Mimir$.eventLoop(Mimir.scala:150)
        at mimir.Mimir$.interactiveEventLoop(Mimir.scala:121)
        at mimir.Mimir$.main(Mimir.scala:112)
        at mimir.Mimir.main(Mimir.scala)
Caused by: java.lang.UnsupportedOperationException: Cannot evaluate expression: row_number()
        at org.apache.spark.sql.catalyst.expressions.Unevaluable$class.eval(Expression.scala:258)
        at org.apache.spark.sql.catalyst.expressions.aggregate.DeclarativeAggregate.eval(interfaces.scala:350)
        at mimir.exec.spark.MimirSpark$$anonfun$registerSparkFunctions$2$$anonfun$apply$1.apply(MimirSpark.scala:245)
        at mimir.exec.spark.MimirSpark$$anonfun$registerSparkFunctions$2$$anonfun$apply$1.apply(MimirSpark.scala:238)
        at mimir.exec.EvalInlined$$anonfun$compileFunction$1.apply(EvalInlined.scala:70)
        at mimir.exec.EvalInlined$$anonfun$compileFunction$1.apply(EvalInlined.scala:70)
        at mimir.exec.EvalInlined$$anonfun$compileForLong$13.apply(EvalInlined.scala:131)
        at mimir.exec.EvalInlined$$anonfun$compileForLong$13.apply(EvalInlined.scala:131)
        at mimir.exec.EvalInlined$$anonfun$compile$2.apply(EvalInlined.scala:45)
        at mimir.exec.EvalInlined$$anonfun$compile$2.apply(EvalInlined.scala:45)
        at mimir.exec.EvalInlined$$anonfun$checkNull$1.apply(EvalInlined.scala:21)
        at mimir.exec.EvalInlined$$anonfun$checkNull$1.apply(EvalInlined.scala:20)
        at mimir.exec.result.LazyRow.apply(LazyRow.scala:17)
        ... 23 more
QUERY took: 157 ms
okennedy commented 4 years ago

Due to optimizations in Mimir, row_number doesn't actually provide consistent row numbering.