Open sukanya-pai opened 3 years ago
try using below,
.option("quote", "\u0000") .option("nullValue",null)
I am also having similar issue and need to get in place of double quote need empty incase there is no value. Please let me know if you got any fix
Using
.option("emptyValue", null) .option("nullValue", null) worked for me
Source : https://stackoverflow.com/questions/62819776/spark-csv-writer-outputs-double-quotes-for-empty-string
Hi, I am trying to send a file from one server to a mainframe server working under SFTP protocol. So I used this package. I am facing the issue that if the field values are blank or empty, I am getting double quotes in place of nothing. I tried using nullValue and emptyValue options with multiple values like "", null,"u/0000", none of them works. I still see double quotes on mainframe server.
Required Output:
If the middle name is blank, then the value should be blank as shown above. but currently, I am getting the below double quotes in the file