search

brimdata / zui

Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
https://www.brimdata.io/download/
Other
1.8k stars 132 forks source link

Add NDJSON/CSV export options #1090

Closed philrz closed 3 years ago

philrz commented 4 years ago

Right now the "Export" button only generates ZNG output. It's not too difficult for users to run that ZNG through zq to generate any number of other supported output formats. However, "vanilla" endpoints were recently added to zqd that return NDJSON and CSV respectively (https://github.com/brimsec/zq/issues/1275, https://github.com/brimsec/zq/issues/1276). As a convenience to the user, it would be great if the Export button offered additional options for NDJSON and CSV that returned the data from these endpoints.

Currently with "Export", the user chooses a filesystem location for output as a file. An additional nice-to-have might be if the user were also offered a single-click button that would instead copy the output directly into the user's paste buffer. This would facilitate quick pivots of CSV data into Excel or Google sheets, or NDJSON data into web-based tools that accept it.

In a group discussion we reached consensus that we'll only want to add this in Brim once we've implemented https://github.com/brimsec/zq/issues/1270 and https://github.com/brimsec/zq/issues/1271 since this will ensure users always get successful CSV output regardless of how much data is returned by their query or if the output crosses heterogeneous record types.

philrz commented 3 years ago

Verified in Brim commit 032da27. The options to export NDJSON or CSV are now added behind the same button that's been used for ZNG export. This makes for easy use in external tools, such as NDJSON for jq or CSV for spreadsheet tools, as shown in the attached video.

https://user-images.githubusercontent.com/5934157/103722118-1e720b80-4f84-11eb-9f57-5b9e72112313.mp4

Note that when CSV is selected, a zqd endpoint is being hit that applies the fuse processor as part of export so that way potentially diverse records can all be represented under a united set of CSV column headers (see brimsec/zq#1271).

Thanks @mason-fish!