natalyjazzviolin
commented
1 year ago @marcelteraflow please edit your initial comment with the feature request template. I will reopen the issue once it's been updated:
Tell us about the problem you're trying to solve
What are you trying to do, and why is it hard? A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you’d like
A clear and concise description of what you want to see happen, or the change you would like to see
Describe the alternative you’ve considered or used
A clear and concise description of any alternative solutions or features you've considered or are using today.
Additional context
Add any other context or screenshots about the feature request here.
Are you willing to submit a PR?
Remove this with your answer :-)
When saving to parquet files to S3, I specified ZSTD compression codec.
Tell us about the problem you're trying to solve
Compression codecs specified in S3 destination aren't reflected in postfixes of filenames.
Describe the solution you’d like
All the files are appended with
.parquetextension, whereas.zstd.parquetwould have be more helpful for downstream work.Describe the alternative you’ve considered or used
Assume the files are snappy encoded.
Additional context
-
Are you willing to submit a PR?
With help