palantir / spark

Palantir Distribution of Apache Spark
Apache License 2.0
67 stars 51 forks source link

[SPARK-27733][CORE] Upgrade Avro to version 1.10.1 #757

Closed LorenzoMartini closed 3 years ago

LorenzoMartini commented 3 years ago

Update Avro dependency to version 1.10.1

To catch up multiple improvements of Avro as well as fix security issues on transitive dependencies.

No

Since there were no API changes required we just run the tests

Closes #31232 from iemejia/SPARK-27733-avro-upgrade.

Authored-by: Ismaël Mejía iemejia@gmail.com Signed-off-by: Dongjoon Hyun dhyun@apple.com

Upstream SPARK-XXXXX ticket and PR link (if not applicable, explain)

[SPARK-27733][CORE] Upgrade Avro to version 1.10.1 https://github.com/apache/spark/pull/31232 or https://github.com/apache/spark/commit/e9e81f798fc2c4a936b967b0e5fb13fd5dfdeb69

This is an "almost" clean cherry pick. Had to modify the test jackson version for this to work, for the rest it is changes from upstream.

What changes were proposed in this pull request?

Bump avro to version 1.10.1

Why are the changes needed?

Avro bump is required for critical bug.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Exising tests