search

hortonworks-spark / spark-llap

Apache License 2.0
101 stars 68 forks source link

Add batch WriteSupport #240

Closed EricWohlstadter closed 6 years ago

EricWohlstadter commented 6 years ago

What changes were proposed in this pull request?

Add batch WriteSupport:

  1. Write DataFrame to ORC temp dir
  2. Use HS2 to "LOAD DATA INPATH" from ORC files to target table
  3. Delete temp dir

How was this patch tested?

UT

EricWohlstadter commented 6 years ago

FYI: Since this PR depends on SPARK-LLAP-229-2, I've set this PR to merge to SPARK-LLAP-229-2, to keep the diff sane.

But the intention is that both will go into branch-2.3-3.0.