apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.4k stars 1.21k forks source link

Add quote-style parameter for CSV options #10669

Open DDtKey opened 6 months ago

DDtKey commented 6 months ago

Is your feature request related to a problem or challenge?

CSV writers usually supports configuration of quote style/mode with the following options:

Sometimes this just need to be controlled, and for now only way to change that is to re-iterate through result file(s) in order to store the content with desired quote style.

You can find such configs in many libraries:

Describe the solution you'd like

Just expose a way to pass the QuoteStyle enum along with other properties like quote, delimiter and etc (as part of CsvOptions). However, need to keep in mind that the configuration only makes sense for writers, not readers.

That shouldn't be an issue to support, because datafusion relies on arrow-csv which uses csv crate under the hood.

Describe alternatives you've considered

No response

Additional context

No response

DDtKey commented 6 months ago

I think this might be labeled with good first issue, there are links to the code that needs to be changed and it is also possible to write sqllogictest similar to https://github.com/apache/datafusion/pull/10671.