search

kite-sdk / kite

Kite SDK
http://kitesdk.org/docs/current/
Apache License 2.0
394 stars 263 forks source link

CDK-1018: Avoid unnecessary copy in MR output format. #382

Closed rdblue closed 9 years ago

rdblue commented 9 years ago

It appears that this was working around PARQUET-62, which fixed dictionary support when incoming records are reused. Updating to 1.6.0 brings in the Parquet fix.

This also adds a property, kite.copyOutputRecords, that allows users to control whether records should be copied. This defaults to false, but is a good safety valve in case of other bugs like PARQUET-62.

joey commented 9 years ago

+1 after Travis comes back green

rdblue commented 9 years ago

Thanks, Joey!