search

brimdata / zui

Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
https://www.brimdata.io/download/
Other
1.8k stars 132 forks source link

after completing ingest, partial results never appear #607

Closed mikesbrown closed 4 years ago

mikesbrown commented 4 years ago

In some cases, results never appear after ingesting a pcap, although the span of the ingested pcap is known. @philrz reports having seen this locally, and integration test CI started encountering it here https://github.com/brimsec/brim/actions/runs/77513682 .

failure-894de677

Notice above how there is a span with no results.

mikesbrown commented 4 years ago

I should mention too the first time I was aware of this issue in CI was after the zq version bump.

alfred-landrum commented 4 years ago

@philrz and I looked at a repro of this with devtools open to examine the network interactions: In the case we saw, the first PacketPostStatus response had a snapshot_count of 0. This means that no records had yet been written into the spaces zng file.

jameskerr commented 4 years ago

Thanks for the research @philrz and @alfred-landrum . I'll put up a fix for this this morning.

mikesbrown commented 4 years ago

Verified at d95f9312 . I was able to reproduce this reliably on my laptop by doing the following:

$ rm -rf itest/sample.pcap.brim/
$ rm -rf ~/Library/Application\ Support/Brim/
$ for i in $(seq 12); do yes > /dev/null & done
$ npm run electron

The parallel yes(1) were needed to create enough load to slow down the system.

Pull up developer tools and, network tab, and load itest/sample.pcap into Brim

Before, I observed an initial "snapshot_count": 0, proper span values shown in Brim, and no results in the viewer or histogram.

Now, even with "snapshot_count": 0, I see results.

image