search

brimdata / zed

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.34k stars 67 forks source link

Panic when attempting to use "load" operator with zq #5157

Closed philrz closed 2 days ago

philrz commented 3 days ago

Repro is with Zed commit ab603fa.

As reported in a community Slack thread, attempts to use the load operator in a Zed query executed via zq causes a panic.

$ zq -version
Version: v1.16.0-3-gab603fa7

$ echo '{"foo": "bar"}' | zq 'load foo' -
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x22308cc]

goroutine 1 [running]:
github.com/brimdata/zed/lake.(*Root).PoolID(0xc00051a90c?, {0x2940750?, 0xc0007d0320?}, {0xc00051a90c, 0x3})
    /Users/phil/work/zed/lake/root.go:245 +0x4c
github.com/brimdata/zed/compiler/data.(*Source).PoolID(0xc0005271e8, {0x2940750, 0xc0007d0320}, {0xc00051a90c, 0x3})
    /Users/phil/work/zed/compiler/data/source.go:47 +0xe7
github.com/brimdata/zed/compiler/semantic.(*analyzer).semOp(0xc0007ca310, {0x2939e50?, 0xc0007ca2a0}, {0x0, 0x0, 0x0})
    /Users/phil/work/zed/compiler/semantic/op.go:776 +0x2bf2
github.com/brimdata/zed/compiler/semantic.(*analyzer).semSeq(...)
    /Users/phil/work/zed/compiler/semantic/op.go:28
github.com/brimdata/zed/compiler/semantic.AnalyzeAddSource({0x2940750, 0xc0007d0320}, {0xc0007b8680, 0x1, 0x247d6c0?}, 0x4a8fdf88?, 0xc00041fa20?)
    /Users/phil/work/zed/compiler/semantic/analyzer.go:30 +0x7d
github.com/brimdata/zed/compiler.NewJob(0xc0007d0370, {0xc0007b8330?, 0x1011285?, 0x48?}, 0xc0005271e8, 0xc0007d0301?)
    /Users/phil/work/zed/compiler/job.go:46 +0x7e
github.com/brimdata/zed/compiler.(*fsCompiler).NewQuery(0x2940750?, 0xc0004c2aa0?, {0xc0007b8330?, 0x3d73658?, 0x3d683c8?}, {0xc0007b8540, 0x1, 0x1})
    /Users/phil/work/zed/compiler/file.go:25 +0x45
github.com/brimdata/zed/runtime.CompileQuery({0x2940750?, 0xc0004c2aa0?}, 0xc0007d0190, {0x2940638, 0xc00007a0d0}, {0xc0007b8330, 0x1, 0x1}, 0xc0007b4060, {0xc0007b8540, ...})
    /Users/phil/work/zed/runtime/compiler.go:49 +0x126
github.com/brimdata/zed/cli/zq.(*Command).Run(0xc0004ce1e0, {0xc000040730, 0x2, 0x2})
    /Users/phil/work/zed/cli/zq/command.go:157 +0x6ca
github.com/brimdata/zed/pkg/charm.path.run({0xc0006302f8?, 0x1, 0x1}, {0xc000040730?, 0x2, 0x0?})
    /Users/phil/work/zed/pkg/charm/path.go:11 +0x79
github.com/brimdata/zed/pkg/charm.(*Spec).ExecRoot(0x100874b?, {0xc000040730, 0x2, 0x2})
    /Users/phil/work/zed/pkg/charm/charm.go:63 +0x3f
main.main()
    /Users/phil/work/zed/cmd/zq/main.go:11 +0x5b

We know this is an unsupported combination of course, but a helpful error message should surely be shown instead.

philrz commented 2 days ago

Verified in Zed commit f497079.

Now when an attempt is made to use the load operator in a Zed program invoked via zq, an error message is shown.

$ zq -version
Version: v1.16.0-6-gf4970799

$ echo '{"foo": "bar"}' | zq 'load foo' -
load operator cannot be used without a lake at line 1, column 1:
load foo
~~~~~~~~

The load docs have also been updated to clarify this.

Thanks @mattnibs!