search

brimdata / zed

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.38k stars 67 forks source link

Panic when attempting to write primitive Zed values as Zeek TSV #5107

Open philrz opened 5 months ago

philrz commented 5 months ago

tl;dr

$ zq -version
Version: v1.15.0-7-g0db1b737

$ echo 'error("hello")' | zq -f zeek -
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x116cb73]

goroutine 1 [running]:
github.com/brimdata/zed.Value.Fields({{0x28c8688?, 0xc000713e18?}, 0xc00072e000?, 0xc00063b3b0?})
    /Users/phil/work/zed/value.go:354 +0x33
github.com/brimdata/zed/runtime/sam/expr.(*Flattener).Flatten(0xc000508d20, {{0x28c8688?, 0xc000713e18?}, 0xc00072e000?, 0xc00041fa68?})
    /Users/phil/work/zed/runtime/sam/expr/flattener.go:67 +0x8e
github.com/brimdata/zed/zio/zeekio.(*Writer).Write(0xc00053cb60, {{0x28c8688, 0xc000713e18}, 0xc00072e000, 0x5})
    /Users/phil/work/zed/zio/zeekio/writer.go:33 +0x51
github.com/brimdata/zed/zbuf.WriteBatch({0x4ab262b0, 0xc00053cb60}, {0x28d5d30?, 0xc00051aa40?})
    /Users/phil/work/zed/zbuf/batch.go:41 +0x84
github.com/brimdata/zed/zbuf.CopyPuller({0x4ab262b0, 0xc00053cb60}, {0x28c2380, 0xc000541440})
    /Users/phil/work/zed/zbuf/batch.go:166 +0x76
github.com/brimdata/zed/cli/zq.(*Command).Run(0xc0004caa80, {0xc0000400b0, 0x1, 0x1})
    /Users/phil/work/zed/cli/zq/command.go:162 +0x79e
github.com/brimdata/zed/pkg/charm.path.run({0xc0005a62f8?, 0x1, 0x1}, {0xc0000400b0?, 0x1, 0x0?})
    /Users/phil/work/zed/pkg/charm/path.go:11 +0x79
github.com/brimdata/zed/pkg/charm.(*Spec).ExecRoot(0x100874b?, {0xc000040090, 0x3, 0x3})
    /Users/phil/work/zed/pkg/charm/charm.go:63 +0x3f
main.main()
    /Users/phil/work/zed/cmd/zq/main.go:11 +0x5b

$ echo '"hello"' | zq -f zeek -
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x2194499]

goroutine 1 [running]:
github.com/brimdata/zed/zio/zeekio.(*Writer).writeHeader(0xc0000b9040, {{0x28c84d0?, 0x36dfb60?}, 0xc000780000?, 0xc00041fa68?}, {0x0, 0x0})
    /Users/phil/work/zed/zio/zeekio/writer.go:91 +0x3d9
github.com/brimdata/zed/zio/zeekio.(*Writer).Write(0xc0000b9040, {{0x28c84d0, 0x36dfb60}, 0xc000780000, 0x5})
    /Users/phil/work/zed/zio/zeekio/writer.go:39 +0x185
github.com/brimdata/zed/zbuf.WriteBatch({0x4ab38298, 0xc0000b9040}, {0x28d5d30?, 0xc00074a940?})
    /Users/phil/work/zed/zbuf/batch.go:41 +0x84
github.com/brimdata/zed/zbuf.CopyPuller({0x4ab38298, 0xc0000b9040}, {0x28c2380, 0xc00076d170})
    /Users/phil/work/zed/zbuf/batch.go:166 +0x76
github.com/brimdata/zed/cli/zq.(*Command).Run(0xc000139a40, {0xc0000400b0, 0x1, 0x1})
    /Users/phil/work/zed/cli/zq/command.go:162 +0x79e
github.com/brimdata/zed/pkg/charm.path.run({0xc0006a42f8?, 0x1, 0x1}, {0xc0000400b0?, 0x1, 0x0?})
    /Users/phil/work/zed/pkg/charm/path.go:11 +0x79
github.com/brimdata/zed/pkg/charm.(*Spec).ExecRoot(0x100874b?, {0xc000040090, 0x3, 0x3})
    /Users/phil/work/zed/pkg/charm/charm.go:63 +0x3f
main.main()
    /Users/phil/work/zed/cmd/zq/main.go:11 +0x5b

Details

Repro is with Zed commit 0db1b73.

I totally expected that Zed tooling would refuse to output primitive Zed values in Zeek TSV since as far as I know the Zeek TSV log format assumes everything must fit into a record. That said, we'd ideally not want to panic.

I happened to try the error value first, then found the simpler string value also created a panic. Since the panics look a bit different I've included both.