search

brimdata / zui

Zui is a powerful desktop application for exploring and working with data. The official front-end to the Zed lake.
https://www.brimdata.io/download/
Other
1.79k stars 131 forks source link

Histogram not drawn after importing a larger pcap #1656

Closed philrz closed 3 years ago

philrz commented 3 years ago

Repro is with Brim commit 207cb69 which uses Zed commit 1d16c1b. The issue actually did not appear at a Brim-side code change but rather when the Zed dependency in package.json was updated to that Zed commit. It's not clear to me if the problem is because some Zed bug unexpectedly broke Brim or if we just missed updating Brim to adapt to a new Zed behavior, so I've flipped a coin and opened this on the Brim side to start.

As shown in the attached video, if I import a larger pcap such as https://storage.googleapis.com/brimsec-releases/wrccdc.pcap, the histogram never draws and there's an error dump visible in DevTools. With smaller pcaps I've seen the histogram draw ok.

https://user-images.githubusercontent.com/5934157/118733121-89fc9d00-b7f0-11eb-986a-18c1b71aaada.mp4

Here's a text dump of the error shown in the video:

POST http://localhost:9867/search?format=zjson net::ERR_FAILED 200 (OK)
_callee2$    @    /Users/phil/work/bri…tcher/fetcher.js:84
tryCatch    @    /Users/phil/work/bri…ntime/runtime.js:45
invoke    @    /Users/phil/work/bri…time/runtime.js:281
prototype.<computed>    @    /Users/phil/work/bri…time/runtime.js:106
asyncGeneratorStep    @    /Users/phil/work/bri…tcher/fetcher.js:36
_next    @    /Users/phil/work/bri…tcher/fetcher.js:38
(anonymous)    @    /Users/phil/work/bri…tcher/fetcher.js:38
(anonymous)    @    /Users/phil/work/bri…tcher/fetcher.js:38
stream    @    /Users/phil/work/bri…cher/fetcher.js:116
_callee3$    @    /Users/phil/work/bri…ws/getZealot.js:152
tryCatch    @    /Users/phil/work/bri…ntime/runtime.js:45
invoke    @    /Users/phil/work/bri…time/runtime.js:281
prototype.<computed>    @    /Users/phil/work/bri…time/runtime.js:106
asyncGeneratorStep    @    /Users/phil/work/bri…ows/getZealot.js:30
_next    @    /Users/phil/work/bri…ows/getZealot.js:32
Promise.then (async)        
asyncGeneratorStep    @    /Users/phil/work/bri…ows/getZealot.js:30
_next    @    /Users/phil/work/bri…ows/getZealot.js:32
(anonymous)    @    /Users/phil/work/bri…ows/getZealot.js:32
(anonymous)    @    /Users/phil/work/bri…ows/getZealot.js:32
wrappedStream    @    /Users/phil/work/bri…ws/getZealot.js:163
search    @    /Users/phil/work/bri…zealot/zealot.js:63
(anonymous)    @    /Users/phil/work/bri…ws/search/mod.js:60
(anonymous)    @    /Users/phil/work/bri…unk/lib/index.js:11
dispatch    @    /Users/phil/work/bri…ux/lib/redux.js:644
(anonymous)    @    /Users/phil/work/bri…togram-search.js:62
(anonymous)    @    /Users/phil/work/bri…unk/lib/index.js:11
(anonymous)    @    /Users/phil/work/bri…chHeaderChart.js:47
commitHookEffectListMount    @    /Users/phil/work/bri…evelopment.js:19731
commitLifeCycles    @    /Users/phil/work/bri…evelopment.js:19787
commitLayoutEffects    @    /Users/phil/work/bri…evelopment.js:22803
callCallback    @    /Users/phil/work/bri….development.js:188
invokeGuardedCallbackDev    @    /Users/phil/work/bri….development.js:237
invokeGuardedCallback    @    /Users/phil/work/bri….development.js:292
commitRootImpl    @    /Users/phil/work/bri…evelopment.js:22541
unstable_runWithPriority    @    /Users/phil/work/bri….development.js:653
runWithPriority$1    @    /Users/phil/work/bri…evelopment.js:11039
commitRoot    @    /Users/phil/work/bri…evelopment.js:22381
finishSyncRender    @    /Users/phil/work/bri…evelopment.js:21807
performSyncWorkOnRoot    @    /Users/phil/work/bri…evelopment.js:21793
(anonymous)    @    /Users/phil/work/bri…evelopment.js:11089
unstable_runWithPriority    @    /Users/phil/work/bri….development.js:653
runWithPriority$1    @    /Users/phil/work/bri…evelopment.js:11039
flushSyncCallbackQueueImpl    @    /Users/phil/work/bri…evelopment.js:11084
flushSyncCallbackQueue    @    /Users/phil/work/bri…evelopment.js:11072
batchedUpdates$1    @    /Users/phil/work/bri…evelopment.js:21862
notify    @    /Users/phil/work/bri…/Subscription.js:25
notifyNestedSubs    @    /Users/phil/work/bri…/Subscription.js:98
handleChangeWrapper    @    /Users/phil/work/bri…Subscription.js:103
dispatch    @    /Users/phil/work/bri…ux/lib/redux.js:228
(anonymous)    @    /Users/phil/work/bri…GlobalContext.js:17
(anonymous)    @    /Users/phil/work/bri…unk/lib/index.js:14
dispatch    @    /Users/phil/work/bri…ux/lib/redux.js:644
_callee6$    @    /Users/phil/work/bri…/ingestFiles.js:290
tryCatch    @    /Users/phil/work/bri…ntime/runtime.js:45
invoke    @    /Users/phil/work/bri…time/runtime.js:281
prototype.<computed>    @    /Users/phil/work/bri…time/runtime.js:106
asyncGeneratorStep    @    /Users/phil/work/bri…s/ingestFiles.js:40
_next    @    /Users/phil/work/bri…s/ingestFiles.js:42
Promise.then (async)        
asyncGeneratorStep    @    /Users/phil/work/bri…s/ingestFiles.js:40
_next    @    /Users/phil/work/bri…s/ingestFiles.js:42
(anonymous)    @    /Users/phil/work/bri…s/ingestFiles.js:42
(anonymous)    @    /Users/phil/work/bri…s/ingestFiles.js:42
onDetailUpdate    @    /Users/phil/work/bri…/ingestFiles.js:301
_callee4$    @    /Users/phil/work/bri…imcap-plugin.js:418
tryCatch    @    /Users/phil/work/bri…ntime/runtime.js:45
invoke    @    /Users/phil/work/bri…time/runtime.js:281
prototype.<computed>    @    /Users/phil/work/bri…time/runtime.js:106
asyncGeneratorStep    @    /Users/phil/work/bri…rimcap-plugin.js:54
_next    @    /Users/phil/work/bri…rimcap-plugin.js:56
(anonymous)    @    /Users/phil/work/bri…rimcap-plugin.js:56
(anonymous)    @    /Users/phil/work/bri…rimcap-plugin.js:56
(anonymous)    @    /Users/phil/work/bri…imcap-plugin.js:439
emit    @    events.js:315
addChunk    @    _stream_readable.js:295
readableAddChunk    @    _stream_readable.js:271
Readable.push    @    _stream_readable.js:212
onStreamRead    @    internal/stream_base_commons.js:186

As mentioned above, I've confirmed this problem first showed up when the Zed commit in Brim's package.json was advanced to Zed commit 1d16c1b6ae24c20210683c533905598239d37d80, which is associated with brimdata/zed#2731 (cc: @mccanne). If I update that dependency in package.json to the prior Zed commit d5b407d05f6c9aa82c7cb87accf2aa083600b980, the histogram draws ok and the error dump does not appear in DevTools.

philrz commented 3 years ago

The root cause of this was fixed on the Zed side via brimdata/zed#2745. However, while studying this symptom, @nwt spotted some unexpected redundant requests coming from Brim. These were shown to @jameskerr and he agreed to take a closer look at them. #1666 has been opened to track that. Closing this one.