Closed philrz closed 4 years ago
I believe upcoming changes spawned from work in #1183 are going to alter how we store the data & availability of indexes, so I'm moving this to the backlog.
I checked in on this one, since things have changed a bit and I wanted to see if this coincidentally got fixed. The tl;dr is that it's still with us.
As of zar
commit d8f8855
that came with overhaul PR #1330, the indexes
section of zar.json
is now gone. Following the original repro steps and stopping just before the zar rm custom.zng
step, it looks like:
{
"version": 0,
"data_path": ".",
"data_sort_direction": false,
"log_size_threshold": 25000000
}
But then the crash is similar to what we saw before:
$ zar rm custom.zng
file:///Users/phil/logs/zd/20180324/d-1iR05l4KpUnpTqHlCHqu3wdZznE.zng.zar/custom.zng: removed
file:///Users/phil/logs/zd/20180324/d-1iR05jwpKeMyIHTkvcrSfrr7Ceg.zng.zar/custom.zng: removed
file:///Users/phil/logs/zd/20180324/d-1iR05uHdW2v4iXIyCu7CbEgdOGe.zng.zar/custom.zng: removed
file:///Users/phil/logs/zd/20180324/d-1iR05o9uFc8DHAjcrYQ4fLRLiHh.zng.zar/custom.zng: removed
$ zar find -z -x custom.zng 10.164.94.120 | zq -t -
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x16b98d6]
goroutine 32 [running]:
github.com/brimsec/zq/microindex.(*Reader).Path(...)
/Users/phil/work/zq/microindex/reader.go:134
github.com/brimsec/zq/archive.search(0x1aaf9e0, 0xc0002d6900, 0xc0000e0730, 0xc00009e8a0, 0xc0000dcaa0, 0x4, 0x0, 0x0, 0x0, 0xc0000dcaa7, ...)
/Users/phil/work/zq/archive/finder.go:128 +0x1a6
github.com/brimsec/zq/archive.Find.func2.1(0xc00009e8a0, 0x1aaf9e0, 0xc0002d6900, 0xc0000b9e20, 0xc0000dcaa0, 0x4, 0x0, 0x0, 0x0, 0xc0000dcaa7, ...)
/Users/phil/work/zq/archive/finder.go:101 +0x191
created by github.com/brimsec/zq/archive.Find.func2
/Users/phil/work/zq/archive/finder.go:99 +0x189
However, more recently, starting with commit 56ed80d
, the zar index
phase of the repro steps no longer works due to #1415.
I now realize that what remains of this problem is super simple to describe, so all the history I've documented here isn't serving us well. I've opened #1419 to capture the essence of what's now wrong, and will close this one.
Repro is with
zq
commit41a4641
and steps derived from thezar
README.Staring with an empty
$ZAR_ROOT
, if I follow the steps:The microindex files are removed as expected:
But I still see an index entry in
zar.json
:So now I can do:
Granted, by a strict read of
zar help rm
, it's doing what it promised. But I assume the additional cleanup is necessary to avoid such crashes.