apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.89k stars 4.27k forks source link

Beam Row Avro conversion: fixed and bytes decimals #21226

Open damccorm opened 2 years ago

damccorm commented 2 years ago

An Avro decimal logical type annotates Avro bytes or fixed types.

Current Row to Avro conversion is limited to bytes type and in addition hardcodes precision to MAX_INT and scale to 0. 

I have attached a patch that adds support for decimal bytes and fixed types.

I could think of 2 possible ways to solve this:

I felt that overhauling Beam schema types might not be a good idea. Instead I went with option 2.

Passes ./gradlew check

 

 

Imported from Jira BEAM-12877. Original Jira may contain additional context. Reported by: kegelink.

gabrywu commented 1 year ago

any updates?