search

Azure / azure-sqldb-spark

This project provides a client library that allows Azure SQL DB or SQL Server to act as an input source or output sink for Spark jobs.
MIT License
76 stars 52 forks source link

Migrate class and pom.xml to support Spark 3.0.0-preview2 #66

Closed theerapan closed 4 years ago

theerapan commented 4 years ago

Spark V.3 was migrated Parquet to File Data Source V2 and this libs is still using Spark 2 and the org.apache.spark.sql.execution.datasources.parquet.ParquetFileFormat, and this make the spark confusing about source type and have to specify full path of source class in every read/write.

I've migrating the library to support Spark 3 with the org.apache.spark.sql.execution.datasources.v2.parquet.ParquetDataSourceV2.

msftclas commented 4 years ago

CLA assistant check
All CLA requirements met.

arvindshmicrosoft commented 4 years ago

Thank you very much for your interest and contribution, @theerapan . Unfortunately, this change is non-trivial and considering that the project is not actively maintained, we are unable to merge the contribution. The newer connector has related activity on supporting Spark 3.0. Please evaluate and if required, we welcome your contributions to that project. Closing this PR.