apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
447 stars 100 forks source link

Limit with offset return incorrect results #358

Closed viirya closed 2 weeks ago

viirya commented 2 weeks ago

Describe the bug

We ignore offset parameter for local and global limit operators which cause incorrect result in SQLQueryTestSuite.scala.

[info] - subquery/in-subquery/in-limit.sql *** FAILED *** (2 seconds, 757 milliseconds)
[info]   subquery/in-subquery/in-limit.sql
[info]   Expected "val1[]d  NULL    16  19  17.0    25...", but got "val1[a 16  12  10  15.0    20.0    2000    2014-07-04 01:01:00 2014-07-04
[info]   val1c  8   16  19  17.0    25.0    2600    2014-05-04 01:02:00.001 2014-05-05
[info]   val1]d NULL    16  19  17.0    25..." Result did not match for query #9
[info]   SELECT *
[info]   FROM   t1
[info]   WHERE  t1c IN (SELECT t2c
[info]                  FROM   t2
[info]                  WHERE  t2b >= 8
[info]                  LIMIT  2
[info]                  OFFSET 2)
[info]   LIMIT 4
[info]   OFFSET 2 (SQLQueryTestSuite.scala:495)

Found this when debugging test failure in the PR #250.

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response