search

jghoman / haivvreo

Hive + Avro. Serde for working with Avro in Hive
Apache License 2.0
59 stars 27 forks source link

Provide option to compress output file when writing to Avro #2

Closed jghoman closed 13 years ago

jghoman commented 13 years ago

Currently we don't compress tables stored into Avro. We should provide serde properties for specifying codec and compression level.

koertkuipers commented 13 years ago

i just created a table in hive and populated it. looking at the byte contents of some of the files on hdfs i see this:

Objavro.codecdeflateavro.schema {"type":"record","name":"test","namespace":"com.linkedin.haivvreo","fields":[{"name":"

so it seems to be using compression

jghoman commented 13 years ago

Yeah, Tom had a patch to add this earlier and the issue was left open. Thanks for catching it!