vertica / spark-connector

This component acts as a bridge between Spark and Vertica, allowing the user to either retrieve data from Vertica for processing in Spark, or store processed data from Spark into Vertica.
Apache License 2.0
20 stars 23 forks source link

Option for truncation in overwrite mode #510

Closed ai-bq closed 1 year ago

ai-bq commented 1 year ago

Summary

Some users require the option of truncating a table rather than dropping it in overwrite mode to retain table privileges.

Description

Truncate table was added to table utilities as well as the option of truncation through the dataframe writing options.

Related Issue

Closes #507

Additional Reviewers

@alexey-temnikov @alexr-bq @jonathanl-bq @jeremyp-bq

codecov[bot] commented 1 year ago

Codecov Report

Merging #510 (5c7565e) into main (00e7f37) will decrease coverage by 0.00%. The diff coverage is 90.90%.

@@            Coverage Diff             @@
##             main     #510      +/-   ##
==========================================
- Coverage   87.52%   87.52%   -0.01%     
==========================================
  Files          44       44              
  Lines        1988     1996       +8     
  Branches      110      124      +14     
==========================================
+ Hits         1740     1747       +7     
- Misses        248      249       +1     
Flag Coverage Δ
unittests 87.52% <90.90%> (-0.01%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...n/scala/com/vertica/spark/config/WriteConfig.scala 100.00% <ø> (ø)
.../vertica/spark/datasource/core/DSConfigSetup.scala 88.07% <83.33%> (-0.14%) :arrow_down:
...e/core/VerticaDistributedFilesystemWritePipe.scala 88.20% <100.00%> (+0.05%) :arrow_up:
...cala/com/vertica/spark/util/table/TableUtils.scala 82.17% <100.00%> (+0.36%) :arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more