If parquet.path.outputcommitter.enabled is true
then it uses the PathOutputCommitterFactory mechanism to dynamically
choose a committer for the output path.
Such committers do not generate summary files; a warning
about this is printed when appropriate
This significantly simplifies writing to s3/azure/gcs
though committers which commit correctly and efficiently
to the target stores.
Jira
[X] My PR addresses the following Parquet Jira issues and references
them in the PR title. For example, "PARQUET-1234: My Parquet PR"
[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:
No tests yet
Commits
[X] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines
from "How to write a good git commit message":
Subject is separated from body by a blank line
Subject is limited to 50 characters (not including Jira issue reference)
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"
Style
[X] My contribution adheres to the code style guidelines and Spotless passes.
To apply the necessary changes, run mvn spotless:apply -Pvector-plugins
Documentation
[ ] In case of new functionality, my PR adds documentation that describes how to use it.
All the public functions and the classes in the PR contain Javadoc that explain what it does
If
parquet.path.outputcommitter.enabled
is true then it uses thePathOutputCommitterFactory
mechanism to dynamically choose a committer for the output path. Such committers do not generate summary files; a warning about this is printed when appropriateThis significantly simplifies writing to s3/azure/gcs though committers which commit correctly and efficiently to the target stores.
Jira
Tests
No tests yet
Commits
Style
mvn spotless:apply -Pvector-plugins
Documentation