AbsaOSS / ABRiS

Avro SerDe for Apache Spark structured APIs.
Apache License 2.0
230 stars 75 forks source link

Abris #239 part 1 - Upgrade Confluent platform to 6.2.1 and Avro to 1.10.2 #244

Closed gundersborg closed 2 years ago

gundersborg commented 2 years ago

Upgrade to Confluent 6.2.1 and Avro 1.10.2. First part of a fix for #239.

I ran the tests for Spark 3.0 and 3.1 using Scala 2.12 as well as for Spark 2.3 and 2.4 using Scala 2.11 - all green.

My company has had a kafka->spark job running in prod since last week on Spark 3.1, and so far we haven't observed any issues.