search

apache / parquet-java

Apache Parquet Java
https://parquet.apache.org/
Apache License 2.0
2.65k stars 1.41k forks source link

Parquet 1.14.2-RC1 breaks compat with Avro 1.7, 1.8, 1.9 #2992

Closed clairemcginty closed 3 months ago

clairemcginty commented 3 months ago

Describe the bug, including details regarding any error messages, version, and platform.

AvroSchemaConverter references Avro logical type converters that aren't available in Avro versions below 1.10: See more here: https://github.com/apache/parquet-java/pull/1115/files#r1721966099

For reference, these are the local TIMESTAMP converters available in Avro 1.10+: https://github.com/apache/avro/blob/release-1.10.0/lang/java/avro/src/main/java/org/apache/avro/LogicalTypes.java#L172-L194

Whereas in 1.9.2, only timestampMillis/timestampMicros area available: https://github.com/apache/avro/blob/release-1.9.2/lang/java/avro/src/main/java/org/apache/avro/LogicalTypes.java#L158-L168

Component(s)

Avro

wgtmac commented 3 months ago

@Fokko Should we include this in the 1.14.2?

Fokko commented 3 months ago

@wgtmac Yes, looks like it 👍