search

spotify / spark-bigquery

Google BigQuery support for Spark, SQL, and DataFrames
Apache License 2.0
155 stars 52 forks source link

Upgrade to databricks spark-avro 4.0.0 (officially supporting Spark 2.2) #47

Closed martinstuder closed 6 years ago

martinstuder commented 6 years ago

Upgrade to databricks spark-avro 4.0.0 which officially supports Spark 2.2. This also updates the copy of SchemaConverters.scala and applies the same changes as before - although it would be nice to get rid of this ultimately.

nevillelyh commented 6 years ago

Thanks!

liyinan926 commented 6 years ago

@nevillelyh When are you going to cut a new release for Spark 2.2 that includes this change?

nevillelyh commented 6 years ago

Kinda busy right now but probably sometime next week.

nevillelyh commented 6 years ago

Just published. Waiting for the site to update. https://spark-packages.org/staging?id=2311 https://spark-packages.org/staging?id=2312

nevillelyh commented 6 years ago

Failed the 1st attempt. 2nd try worked. It's released now. https://spark-packages.org/staging?id=2313 https://spark-packages.org/staging?id=2314

liyinan926 commented 6 years ago

Awesome! Thanks!