Closed nurzhannogerbek closed 5 years ago
@NogerbekNurzhan
Could you please try cleaning "com.springml" % "sftp.client" % "1.0.3" from local repository and run the code again?
@samuel-pt Thank you for your answer! I removed com.springml.sftp.client
library as you recommended. Also I removed codec
option. Final code looks like this:
df.write.format("com.springml.spark.sftp")
.option("host", "XXXX")
.option("username", "XXXX")
.option("password", "XXXX")
.option("fileType", "csv")
.option("delimiter", ";")
.option("charset", "windows-1251")
.mode("append")
.save("/reports/daily.csv")
After rebuilding the project I notice that sbt reinstall com.springml.sftp.client
library. I run the project without previous error now.
I notice only one problem. For some reason non-Latin letters (in my case cyrillic) are displayed incorrectly in final csv file. Inside that csv file I see strange symbols: Село
. Why charset
option don't work?
@NogerbekNurzhan - Sorry, we got into something more priority. If possible can you create PR with the required changes?
In my Spark application I need to convert
DataFrame
to.csv
file and put it to remote SFTP server. I decided to usespark-sftp
library for this task.My sbt file looks like this:
I compile sbt file without any error. When I try to test next code it raise error.
ERROR:
What the reason of the problem?