search

dbro / csvquote

Enables common unix utlities like cut, awk, wc, head to work correctly with csv data containing delimiters and newlines
MIT License
446 stars 24 forks source link

Add option to flush buffer #14

Closed jarnos closed 7 years ago

jarnos commented 7 years ago

Sometimes you may want to pipe encoding and then you need only some parts of its output and decode that and use the result as part of output (which may include things that were not in the original file passed for csvquote). Normally this won't work due to buffering, but flushing makes it possible.

I have added such an option to awk implementation here https://github.com/jarnos/csvquote/commit/2d598c0510ae1713dfb455b9f02890c25689fc6f

dbro commented 7 years ago

Hi Jarno- I think it is possible and preferable to use a different command outside of csvquote to handle buffering settings. For example, instead of

> csvquote inputfile | cut -f2 | csvquote -u

if no buffering of stdin and stdout is desired, then this can be modified to be:

> stdbuf -i0 -o0 csvquote inputfile | cut -f2 | csvquote -u

Does that address your situation? If not, please include more information about the data and how you want to work with it, in terms of input and output.