Open alaukikharsh opened 2 years ago
getting error no such file while writing a df to sftp path using spark. how can i write a dummy df to sftp path instead? what am i doing wrong here?
val DF = spark.sql("""select 1,2,3 as a""") DF.show(10) DF.coalesce(1).write. format("com.springml.spark.sftp"). option("host", "correct 1"). option("username", "correct 1"). option("pem", "correct 1"). option("fileType", "csv"). option("codec", "bzip2"). save(/PRODUCTION/SAMPLE/a.csv)
getting error no such file while writing a df to sftp path using spark. how can i write a dummy df to sftp path instead? what am i doing wrong here?