wireservice / csvkit

A suite of utilities for converting to and working with CSV, the king of tabular file formats.
https://csvkit.readthedocs.io
MIT License
6.03k stars 603 forks source link

FR(csvformat): support NULL line terminator #1196

Closed gregorias closed 1 year ago

gregorias commented 1 year ago

csvformat supports a custom terminators, but it's impossible to pass NULL as a line terminator. Try `csvformat -M \x00' (Fish shell), it will think the terminator is empty.

This feature would be useful, because NULL is often used as a safe delimiter as it can't occur naturally, e.g., filenames can contain any character except NULLs. Xargs can use -0 to split its input.

In particular, I can't figure out how to safely process entries with newlines in my plugin, https://github.com/gregorias/bookmarks.fish, without this feature.

This feature would probably mean adding a special flag for it, as NULL can't be passed in argvs.

jpmckinney commented 1 year ago

I think logic around NULL bytes is probably in some of the Python standard libraries.

NULL typically means "stop reading" across many utilities, so it is very non-standard. (I know xargs uses -0, but that's quite the exceptional case.)

You can use the RS (record separator) byte instead.