search

Azure / azure-sqldb-spark

This project provides a client library that allows Azure SQL DB or SQL Server to act as an input source or output sink for Spark jobs.
MIT License
75 stars 52 forks source link

Incompatible With MSSQL Driver 8.2.2?? #76

Closed devmanhinton closed 4 years ago

devmanhinton commented 4 years ago

Please close this if I am wrong. ReadMe notes this works with mssql driver 6.2+

Screen Shot 2020-05-08 at 11 30 31 AM

Ran with 8.2.2.jre8 and ran into

java.lang.NoSuchMethodError: com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(Lcom/microsoft/sqlserver/jdbc/ISQLServerBulkRecord;)V
    at com.microsoft.azure.sqldb.spark.connect.DataFrameFunctions.com$microsoft$azure$sqldb$spark$connect$DataFrameFunctions$$bulkCopy(DataFrameFunctions.scala:127)
    at com.microsoft.azure.sqldb.spark.connect.DataFrameFunctions$$anonfun$bulkCopyToSqlDB$1.apply(DataFrameFunctions.scala:72)

Downgraded to 7.4.1.jre8 and it works

arvindshmicrosoft commented 4 years ago

Thank you @devmanhinton for your interest and efforts to document your issue. Given the 8.2 release JDBC release is relatively new, I am not aware of any testing that was done with that driver version. So I will close this issue with a promise to update the README to reflect this limitation.

Also, consider evaluating the Apache Spark Connector for SQL Server and Azure SQL which is a newer connector. That connector improves the experience around bulk load, and adds other improvements like Python and R bindings etc. However at this time I am not sure about the supportability for MSSQL JDBC 8.2.2 in that connector, either.