apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
823 stars 163 forks source link

perf: Stop passing Java config map into native createPlan #1101

Open andygrove opened 2 days ago

andygrove commented 2 days ago

Which issue does this PR close?

Part of https://github.com/apache/datafusion-comet/issues/1098

Rationale for this change

We were constructing a Java map containing 5 configs and passing that into native and then extracting those configs. This is expensive compared to just passing the 5 configs as parameters.

This config mechanism also supported passing arbitrary DataFusion configs into the native context but I am not aware of anyone using this feature, and perhaps there is a cleaner way to do this if we want to add it back.

What changes are included in this PR?

Just some refactoring to make createPlan a little more efficient.

How are these changes tested?

Existing tests.

codecov-commenter commented 2 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 34.28%. Comparing base (b74bfe4) to head (3ea852f). Report is 6 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1101 +/- ## ============================================ - Coverage 34.33% 34.28% -0.06% + Complexity 898 890 -8 ============================================ Files 115 115 Lines 42986 42922 -64 Branches 9369 9378 +9 ============================================ - Hits 14761 14717 -44 + Misses 25361 25327 -34 - Partials 2864 2878 +14 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features: