slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
446 stars 34 forks source link

Feature/support quote customized #391

Closed yokofly closed 1 month ago

yokofly commented 1 month ago

fix #389 now we can pass a new option to choose which quote symbol we use. usually the standard CSV prefer double quote, but single quote or other symbol in some specific case applied.

below is test output explain: ./sling run --src-conn LOCAL --src-stream file://cmd/sling/tests/files/test7.csv --src-options '{ delimiter: "|", quote: '\''', escape: "\" }' --stdout

single quote csv source ``` col1|header|col3 data1|'data with single quote \' inside'|data3 'data with pipe | inside'|data2|data3 'data with backslash \\ inside'|'data with \"escaped\" double quotes'|data3 ``` sling std output ``` 6:42AM INF writing to target stream (stdout) col1,header,col3 data1,data with single quote ' inside,data3 data with pipe | inside,data2,data3 data with backslash \\ inside,"data with \""escaped\"" double quotes",data3 6:42AM INF wrote 3 rows to stdout in 0 secs [530 r/s] ```

./sling run --src-conn LOCAL --src-stream file://cmd/sling/tests/files/test8.csv --src-options '{ delimiter: "|", quote: "$", escape: "\" }' --stdout

quote use $ symol csv source ``` col1|header|col3 data1|$data with single quote \' inside$|data3 $data with pipe | inside$|data2|data3 $data with backslash \\ inside$|$data with \"escaped\" double quotes$|data3 ``` sling std output ``` 6:42AM INF writing to target stream (stdout) col1,header,col3 data1,data with single quote \' inside,data3 data with pipe | inside,data2,data3 data with backslash \\ inside,"data with \""escaped\"" double quotes",data3 6:42AM INF wrote 3 rows to stdout in 0 secs [541 r/s] ```