search

brimdata / super

An analytics database that puts JSON and relational tables on equal footing
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.4k stars 67 forks source link

vector query causes "panic: runtime error: index out of range [417993] with length 417993" #5469

Closed philrz closed 16 hours ago

philrz commented 2 weeks ago

Repro is with super commit 411514f.

This is a simplification of the bench3/q5 query.

Test data is s3://brim-sampledata/mgbench/bench3.csup.

$ super -version
Version: v1.18.0-142-g411514fd

$ super dev vector query 'where event_type=="temperature" | cut dt := strftime("%Y-%m-%d", log_time)' bench3.csup
panic: runtime error: index out of range [417993] with length 417993

goroutine 1 [running]:
github.com/brimdata/super/runtime/vam/expr/function.(*Strftime).fastPathLoop(0xc000136018?, 0xc0005d0300, 0xc0005d0060, {0xc07c96c000, 0x660c9, 0x0?})
    /Users/phil/work/super/runtime/vam/expr/function/time.go:58 +0x2d6
github.com/brimdata/super/runtime/vam/expr/function.(*Strftime).fastPath(0xc00051a030, 0xc0005d0270, {0xafb3bd0, 0xc0005d02a0})
    /Users/phil/work/super/runtime/vam/expr/function/time.go:40 +0xd4
github.com/brimdata/super/runtime/vam/expr/function.(*Strftime).Call(0xc00051a030, {0xc00052c520, 0x2, 0x0?})
    /Users/phil/work/super/runtime/vam/expr/function/time.go:25 +0x17e
github.com/brimdata/super/vector.Apply(0xb0?, 0xafb38b8?, {0xc00052c520?, 0x1ac000?, 0xc000055a08?})
    /Users/phil/work/super/vector/apply.go:17 +0xa3
github.com/brimdata/super/runtime/vam/expr.(*Call).Eval(0xc00017b360, {0xafb38b8, 0xc00051c090})
    /Users/phil/work/super/runtime/vam/expr/eval.go:39 +0xf4
github.com/brimdata/super/runtime/vam/expr.(*recordExpr).Eval(0xc0006a1340, {0xafb38b8, 0xc00051c090})
    /Users/phil/work/super/runtime/vam/expr/recordexpr.go:38 +0x134
github.com/brimdata/super/runtime/vam/op.(*Yield).Pull(0xc000522e10, 0x0)
    /Users/phil/work/super/runtime/vam/op/yield.go:33 +0x112
github.com/brimdata/super/runtime/vam.(*Materializer).Pull(0x52ff70c0?, 0xf8?)
    /Users/phil/work/super/runtime/vam/materialize.go:25 +0x27
github.com/brimdata/super/zbuf.(*pullerReader).Read(0xc0000bf580)
    /Users/phil/work/super/zbuf/batch.go:193 +0x85
github.com/brimdata/super/zio.CopyWithContext({0xafb3148, 0xbfe9760}, {0x52ff70c0, 0xc000613ad0}, {0xaf99a40, 0xc0000bf580})
    /Users/phil/work/super/zio/zio.go:154 +0x5a
github.com/brimdata/super/zio.Copy(...)
    /Users/phil/work/super/zio/zio.go:146
github.com/brimdata/super/cmd/super/dev/vector/query.(*Command).Run(0xaf99a20?, {0xc00003e2e0, 0x2, 0xc00003e2e0?})
    /Users/phil/work/super/cmd/super/dev/vector/query/command.go:80 +0x5f4
github.com/brimdata/super/pkg/charm.path.run({0xc000697f40, 0x4, 0x4}, {0xc00003e2e0, 0x2, 0x0?})
    /Users/phil/work/super/pkg/charm/path.go:11 +0x7f
github.com/brimdata/super/pkg/charm.(*Spec).Exec(0xbfaf380, {0xc00003e2b0, 0x5, 0x5})
    /Users/phil/work/super/pkg/charm/charm.go:71 +0x105
main.main()
    /Users/phil/work/super/cmd/super/main.go:41 +0x5b
philrz commented 15 hours ago

Verified in super commit 7c930f2.

The query now runs successfully without panic.

$ super -version
Version: v1.18.0-169-g7c930f2a

$ super dev vector query 'where event_type=="temperature" | cut dt := strftime("%Y-%m-%d", log_time)' bench3.csup
{dt:"2017-09-07"}
...
{dt:"2019-09-19"}

Thanks @mattnibs!