search

brimdata / zui

Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
https://www.brimdata.io/download/
Other
1.8k stars 132 forks source link

Improve error messages from import failures #1070

Closed philrz closed 3 years ago

philrz commented 4 years ago

I've had a couple experiences recently that made me realize that the UX during failed import need improvement.

The first came up when a community user reported what winded up being https://github.com/brimsec/zq/issues/1217. They had a log from a Corelight Sensor that was valid Zeek TSV (sample.tsv.gz) but we had a limitation on the zq side that was preventing us from successfully reading it. This is what the user saw:

image

The reaction in their exact words:

error trying to load Zeek file into Brim v0.16.0. i'm getting this message and not sure why it thinks it's a json log.

Indeed, you can probably blame me for the leading message about JSON, as I was concerned that lots of users would try to ingest JSON and hit problems with the typing config. But I can see why this confused that user.

The other I encountered was when filing https://github.com/brimsec/brim/issues/1068. That involved a pcap format the Brim app was not yet prepared to handle. So in Brim GA tagged v0.16.0 this is what I saw when attempting to import it:

image

For a situation like this or if the user tries to drag a compressed pcap into Brim, they're likely to be confused by log-centric error messages, since they thought they were importing packet data.

Putting these two together, here's my proposal that I think would cover both cases. Basically I think we should be more transparent in explaining the logic that was attempted, since that might help the user understand why it failed. Therefore something like:

The data you've attempt to import was not recognized as any supported packet capture or log format. The errors returned by each log parser:

To go with that, I'd suggest moving the message with the link to JSON ingest docs below the errors returned from zqd.

@orochford also suggested that we might consider holding back the per-attempted-log-format errors initially, such as behind some control where they'd be "expanded" if the user wants to see them. This would help stress the top-line message that the data was invalid for import and the additional detail is only relevant if the user wants to get into debugging. This way we don't overwhelm the user with too much info all at once.

philrz commented 3 years ago

Verified in Brim commit a9bccfe7.

Now when I try to import garbage (such as my /etc/passwd file in this case) I'm shown a message that clearly expresses what was attempted & failed.

image

Thanks @mason-fish!