search

apache / parquet-java

Apache Parquet Java
https://parquet.apache.org/
Apache License 2.0
2.62k stars 1.41k forks source link

ParquetOutputCommitter should propagate errors when writing metadata files #1895

Open asfimport opened 9 years ago

asfimport commented 9 years ago

There's a lot of different ways the output committer can fail, or fail to rollback after failing to write metadata files. We should decide whether metadata files are required, and fatal (I think that's reasonable if the user asked for them), and propagate without squashing exceptions.

Reporter: Alex Levenson / @isnotinvain

PRs and other links:

Note: This issue was originally created as PARQUET-383. Please see the migration documentation for further details.

asfimport commented 9 years ago

Ryan Blue / @rdblue: I think this is a good idea. I'd make the error fatal only if the user opted to use the metadata file. I'd suggest that we not write the metadata file by default, but I don't think that's an option without a major version bump because it could break users.