search

brimdata / zui

Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
https://www.brimdata.io/download/
Other
1.8k stars 132 forks source link

Can't maintain column headers on ZNG with diverse records #873

Closed philrz closed 4 years ago

philrz commented 4 years ago

Per the attached video, as of Brim commit e7b06f3 I can select column headers for records that contain diverse fields, e.g. uri (present in Zeek http but not in dns) and query (present in Zeek dns but not in http). Thanks to the fix in https://github.com/brimsec/zq/issues/852, I can now successfully Export the data to ZNG, then re-import it to a separate tab. The re-imported data starts out with no column headers, which I know to expect because the data shown requires multiple descriptors, and we always start out without column headers in that case. However, if I go into the column picker and start clicking off field names, it successfully adds the headers until I click on the last one, at which point it goes back to showing no headers at all.

Based on what I know of the implementation, I understand why it's doing what it's doing, but since the user was able to see the headers pre-export, it feels like a mismatch that they have no way to see them after exporting and re-importing what the user sees as equivalent data. Do we need to change something related to how "Show All" works to cover this case?

Repro.zip

philrz commented 4 years ago

We discussed this as a group today. The consensus was that it would be a significant improvement if we just add another control in the column picker to Show/Remove Headers. This could be an independent control from which fields are included/excluded from the current view (which is what we have today, which also sometimes has side-effects of showing/removing the headers). We're treating this as a target for the Brim v0.12.0 milestone that's planned to be completed in the next ~2 weeks.

We're not planning to change the current default column-free "wall of text" view.

We also discussed possibly including controls to Select/Unselect the entire set of fields in the column picker, which might make it easier if a user wants to show "all but a couple" of a large set of fields, since this would save the user having to individually click a potentially large number of fields. This was seen as "nice-to-have" and not strictly required in this milestone. But if the person who sits down to design & implement finds it would be easy to include at the same time without significantly increasing the time to deliver, go for it.

philrz commented 4 years ago

Verified in Brim commit fce9a46.

Per the attached video, with the redesigned column picker, now the initial set of columns is defined by entering the picker and clicking "Deselect All" before clicking the four four individual field names. The "Export" operation then works as before. Now when the exported ZNG is re-imported to a new Space, it starts out with no headers (much as we saw in the past), but we're now able to force all four headers to appear by selecting the "Headers: On" option in the picker. That was not previously possible, and is the essence of what we were trying to fix here.

Verify.zip

All that said, other operations attempted during verification yielded some other unexpected beahviors, which are now being pursued via newly-opened bug https://github.com/brimsec/brim/issues/948.

Thanks @mason-fish!