search

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
823 stars 163 forks source link

feat: Hook DataFusion Parquet native scan with Comet execution #1094

Closed viirya closed 2 days ago

viirya commented 6 days ago

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

viirya commented 6 days ago

Basic query is okay now.

But this doesn't completely work for all queries right now. It still gets a few test failures in CometExecSuite. I'm looking into that.

viirya commented 2 days ago

A few of test failures in CometExecSuite are due to partition values are not handled for now. It requires some more changes to this branch. I will go to work on it in other PR.

viirya commented 2 days ago

cc @andygrove

viirya commented 2 days ago

Thanks @mbutrovich