search

springml / spark-sftp

Spark connector for SFTP
Apache License 2.0
100 stars 98 forks source link

Error in writing a csv to sftp path using spark "Exception in thread "main" 2: No such file" #89

Open alaukikharsh opened 2 years ago

alaukikharsh commented 2 years ago

getting error no such file while writing a df to sftp path using spark. how can i write a dummy df to sftp path instead? what am i doing wrong here?

val DF = spark.sql("""select 1,2,3 as a""")
DF.show(10)
DF.coalesce(1).write.
      format("com.springml.spark.sftp").
      option("host", "correct 1").
      option("username", "correct 1").
      option("pem", "correct 1").
      option("fileType", "csv").
      option("codec", "bzip2").
      save(/PRODUCTION/SAMPLE/a.csv)