Is your feature request related to a problem or challenge?
CSV writers usually supports configuration of quote style/mode with the following options:
Always
Necessary
Never
NonNumeric
Sometimes this just need to be controlled, and for now only way to change that is to re-iterate through result file(s) in order to store the content with desired quote style.
Just expose a way to pass the QuoteStyle enum along with other properties like quote, delimiter and etc (as part of CsvOptions). However, need to keep in mind that the configuration only makes sense for writers, not readers.
That shouldn't be an issue to support, because datafusion relies on arrow-csv which uses csv crate under the hood.
requires to update arrow-csv to accept quote-style param (sub-issue for arrow-rs?)
I think this might be labeled with good first issue, there are links to the code that needs to be changed and it is also possible to write sqllogictest similar to https://github.com/apache/datafusion/pull/10671.
Is your feature request related to a problem or challenge?
CSV writers usually supports configuration of quote style/mode with the following options:
Always
Necessary
Never
NonNumeric
Sometimes this just need to be controlled, and for now only way to change that is to re-iterate through result file(s) in order to store the content with desired quote style.
You can find such configs in many libraries:
csv
crate (QuoteStyle
),csv
from python (constants, likeQUOTE_ALL
QuoteMode
)Describe the solution you'd like
Just expose a way to pass the
QuoteStyle
enum along with other properties likequote
,delimiter
and etc (as part ofCsvOptions
). However, need to keep in mind that the configuration only makes sense for writers, not readers.That shouldn't be an issue to support, because
datafusion
relies onarrow-csv
which usescsv
crate under the hood.arrow-csv
to accept quote-style param (sub-issue forarrow-rs
?)WriterBuilder
: https://github.com/apache/arrow-rs/blob/4b5d9bfc958c06fb1ff71d90ba58497e965eff40/arrow-csv/src/writer.rs#L191-L214csv::Writer
: https://github.com/apache/arrow-rs/blob/4b5d9bfc958c06fb1ff71d90ba58497e965eff40/arrow-csv/src/writer.rs#L402-L408datafusion
CsvOptions
: https://github.com/apache/datafusion/blob/ea92ae72f7ec2e941d35aa077c6a39f74523ab63/datafusion/common/src/config.rs#L1554-L1570arrow-csv
: https://github.com/apache/datafusion/blob/ea92ae72f7ec2e941d35aa077c6a39f74523ab63/datafusion/common/src/file_options/csv_writer.rs#L48-L75Describe alternatives you've considered
No response
Additional context
No response