brimdata / zed

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.37k stars 67 forks source link

Revisit text mode output #4595

Open philrz opened 1 year ago

philrz commented 1 year ago

At the time this issue is being filed, Zed is at commit a2626a7.

I sometimes reach for text output mode in the CLI tools such as when grabbing individual values to populate environment variables in my Bash scripts. I recently went to output a time value and was thrown by how it's currently printed as epoch seconds.

$ zq -version
Version: v1.7.0-66-ga2626a7c

$ echo '2023-05-17T17:13:37.711504Z' | zq -f text -
1684343617.711504

The mode has been mostly untouched for years, so this specific example is an artifact of when Zeek-like output seemed like a reasonable default. Things have changed enough that something like undecorated ZSON would probably be more appropriate.

philrz commented 3 weeks ago

A community user recently started making use of -f text, and after seeing their struggles I added some minimal coverage of the format in a new Simplified Text Outputs section of the zq command docs (https://github.com/brimdata/zed/pull/5170).

However, even since then, in another community Slack thread a user reported being confused by the \x09 appearing in their output in place of a tab character in their original data. Those of us that are aware of the project's history know this to be a side effect of Zed's Zeek-y roots, but this would be awkward to disclose to a user as if it's a good thing, further making the case that we revisit the format.

I suspect there's some overlap between this and #5042.