search

amazon-ion / ion-hive-serde

A Apache Hive SerDe (short for serializer/deserializer) for the Ion file format.
Apache License 2.0
28 stars 12 forks source link

Add counter for written bytes in IonOutputFormat.IonRecordWriter #70

Closed jacob-stein closed 2 years ago

jacob-stein commented 2 years ago

Description of changes: This change exposes the bytes written during serialization if a Reporter object is passed in as the Progressable "progress" parameter.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jacob-stein commented 2 years ago

Note: This returns the uncompressed size, and will need to be adjusted for compression. I would advocate adding a "CompressedOutputPosition" counter for the "Serializer" counter group, using the FsDataOutputStream.getpos() function.