search

apache / parquet-java

Apache Parquet Java
https://parquet.apache.org/
Apache License 2.0
2.58k stars 1.4k forks source link

Add Brotli compression to Parquet #1940

Open asfimport opened 8 years ago

asfimport commented 8 years ago

Hi,

Google release a new compression algorithm called Brotli. A new compression algorithm for the internet, as they claim. Currently Firefox and Chrome (both developer editions) already support Brotli, as such as e.g. ngingx. http://google-opensource.blogspot.de/2015/09/introducing-brotli-new-compression.html

I'm the author of jbrotli - the Java bindings for brotli.

I'm curios and doing a survey, if jBrotli is an option for parquet? Feedback is welcome.

Thanks in advance Martin

Environment: all Reporter: Martin W. Kirst Assignee: Ryan Blue / @rdblue

Related issues:

Note: This issue was originally created as PARQUET-521. Please see the migration documentation for further details.

asfimport commented 8 years ago

Ryan Blue / @rdblue: [~marki], thanks for opening this issue! I've actually been testing out Brotli lately with jbrotli and didn't realize there was already an issue for it. I just submitted HADOOP-13126 to add a codec to Hadoop and I'm going to post a PR for this issue soon.