This fixes $43 for azure datalake gen1 and gen2, and for that I need to add some configuration options.
For this to work you need to mount your datalake into databricks cluster.
This is a working sample
Df.write.
format("com.springml.spark.sftp").
option("host", salesforceHost).
option("username", salesforceUser).
option("password", salesforcePw).
option("fileType", "csv").
option("delimiter", ",").
option("azuremountpoint", tempFolder). // /dbfs/mnt/your_mount_point/your_folder
option("templocation", gen2TempFolder"). // same address as temp folder, but with abfss protocol
option("gen","gen2").
save(output_destination)
This fixes $43 for azure datalake gen1 and gen2, and for that I need to add some configuration options. For this to work you need to mount your datalake into databricks cluster. This is a working sample Df.write. format("com.springml.spark.sftp"). option("host", salesforceHost). option("username", salesforceUser). option("password", salesforcePw). option("fileType", "csv"). option("delimiter", ","). option("azuremountpoint", tempFolder). // /dbfs/mnt/your_mount_point/your_folder option("templocation", gen2TempFolder"). // same address as temp folder, but with abfss protocol option("gen","gen2"). save(output_destination)