apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.11k stars 915 forks source link

[SPARK] Avoid trigger execution when getting result schema #6688

Closed wForget closed 1 month ago

wForget commented 2 months ago

:mag: Description

Issue References ๐Ÿ”—

DataFrame.isEmpty may trigger execution again, we should avoid it.

Describe Your Solution ๐Ÿ”ง

Types of changes :bookmark:

Test Plan ๐Ÿงช

Behavior Without This Pull Request :coffin:

Behavior With This Pull Request :tada:

Related Unit Tests


Checklist ๐Ÿ“

Be nice. Be informative.

codecov-commenter commented 2 months ago

Codecov Report

Attention: Patch coverage is 0% with 10 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (7198c72) to head (265f0ec). Report is 19 commits behind head on master.

Files with missing lines Patch % Lines
...kyuubi/engine/spark/operation/SparkOperation.scala 0.00% 5 Missing :warning:
.../kyuubi/engine/spark/operation/ExecutePython.scala 0.00% 2 Missing :warning:
...e/kyuubi/engine/spark/operation/ExecuteScala.scala 0.00% 2 Missing :warning:
...ubi/engine/spark/operation/PlanOnlyStatement.scala 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #6688 +/- ## ====================================== Coverage 0.00% 0.00% ====================================== Files 684 684 Lines 42237 42277 +40 Branches 5755 5765 +10 ====================================== - Misses 42237 42277 +40 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

pan3793 commented 2 months ago

cc @iodone

bowenliang123 commented 1 month ago

Thanks, merged to master (1.10.0) and branch-1.9 (1.9.3).