Azure / azure-sqldb-spark

This project provides a client library that allows Azure SQL DB or SQL Server to act as an input source or output sink for Spark jobs.
MIT License
76 stars 52 forks source link

How to retrieve the number of records affected by the sqlContext.sqlDBQuery() method? #93

Closed FArnaud-Keyrus closed 4 years ago

FArnaud-Keyrus commented 4 years ago

Is there any way to retrieve the number of records affected by the sqlContext.sqlDBQuery() method? I'm losing visibility when running a DML query through the Spark Connector because I can't visualize the number of records affected

arvindshmicrosoft commented 4 years ago

An easier way to do this (an example is given below for an UPDATE) is to directly use JDBC instead:

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import com.microsoft.sqlserver.jdbc.SQLServerDataSource;

val ds = new SQLServerDataSource();
ds.setServerName("SOMESERVER.database.windows.net"); // Replace with your server name
ds.setDatabaseName("SOMEDB"); // Replace with your database
ds.setUser("SOMEUSER"); // Replace with your user name
ds.setPassword("SOMEPASSWORD"); // Replace with your password
ds.setHostNameInCertificate("*.database.windows.net");

val connection = ds.getConnection(); 

val query = "UPDATE someTable SET someCol = someThing"

val pStmt = connection.prepareStatement(query, Statement.RETURN_GENERATED_KEYS);
val affectedRows = pStmt.executeUpdate();
print(affectedRows);

Also, please note that this project is not being actively maintained. Instead, Apache Spark Connector for SQL Server and Azure SQL is now available, with support for Python and R bindings, an easier-to use interface to bulk insert data, and many other improvements. We encourage you to actively evaluate and use the new connector. Thank you.