search

brimdata / super

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.39k stars 64 forks source link

Panic in zng handling of record alias types #794

Closed henridf closed 4 years ago

henridf commented 4 years ago

While working on #758 , I noticed that zng also can't handle aliases on records.

16:04 ~/work/brim/zq(zjson-aliases)
$ cat alias.tzng 
#myrec=record[host:ip]
#1:record[foo:myrec]
1:[[127.0.0.2;]]
16:04 ~/work/brim/zq(zjson-aliases)
$ zq alias.tzng | zq -t -
panic: zng cannot translate type: myrec

goroutine 1 [running]:
github.com/brimsec/zq/zng/resolver.(*Context).TranslateType(0xc000312310, 0x1bb68c0, 0xc0002fd170, 0xc0002cb648, 0x1394158)
    /Users/henridf/work/brim/zq/zng/resolver/context.go:581 +0x2f8
github.com/brimsec/zq/zng/resolver.(*Context).TranslateTypeRecord(0xc000312310, 0xc0002fd1a0, 0x0)
    /Users/henridf/work/brim/zq/zng/resolver/context.go:598 +0xc5
github.com/brimsec/zq/zng/resolver.(*Mapper).Enter(0xc0002e7060, 0x19, 0xc0002fd1a0, 0x0)
    /Users/henridf/work/brim/zq/zng/resolver/mapper.go:26 +0x3d
github.com/brimsec/zq/zio/zngio.(*Reader).Read(0xc0002fd0e0, 0xc0002fd001, 0xc0002fd0e0, 0xc0002cb7b0)
    /Users/henridf/work/brim/zq/zio/zngio/reader.go:83 +0xfa
github.com/brimsec/zq/zio/detector.match(0x1ba20e0, 0xc0002fd0e0, 0x196cf61, 0x3, 0xc0002fd0e0, 0xc0002f3890)
    /Users/henridf/work/brim/zq/zio/detector/reader.go:76 +0x38
github.com/brimsec/zq/zio/detector.NewReaderWithConfig(0x1ba1f20, 0xc0002fcde0, 0xc0002e0540, 0x1975465, 0xa, 0x196d45b, 0x4, 0x0, 0x19a92bd, 0x59, ...)
    /Users/henridf/work/brim/zq/zio/detector/reader.go:56 +0x66f
github.com/brimsec/zq/zio/detector.OpenFromNamedReadCloser(0xc0002e0540, 0x1bab7c0, 0xc0000d6000, 0x1975465, 0xa, 0x196d45b, 0x4, 0x0, 0x19a92bd, 0x59, ...)
    /Users/henridf/work/brim/zq/zio/detector/file.go:149 +0xf0
github.com/brimsec/zq/zio/detector.OpenFile(0xc0002e0540, 0x1975465, 0xa, 0x196d45b, 0x4, 0x0, 0x19a92bd, 0x59, 0x0, 0x1861d20, ...)
    /Users/henridf/work/brim/zq/zio/detector/file.go:74 +0x110
main.(*Command).inputReaders(0xc0001a7180, 0xc0000c01a0, 0x1, 0x1, 0x0, 0x0, 0xd6000000019373e0, 0x1115f01, 0xd668107aa6648996)
    /Users/henridf/work/brim/zq/cmd/zq/zq.go:263 +0x167
main.(*Command).Run(0xc0001a7180, 0xc0000c01a0, 0x1, 0x1, 0x0, 0x0)
    /Users/henridf/work/brim/zq/cmd/zq/zq.go:218 +0x2cf
github.com/mccanne/charm.(*instance).run(0xc0002e6340, 0xc0000c0190, 0x2, 0x2, 0x0, 0x0)
    /Users/henridf/go/pkg/mod/github.com/mccanne/charm@v0.0.3-0.20191224190439-b05e1b7b1be3/instance.go:53 +0x28c
github.com/mccanne/charm.(*Spec).ExecRoot(0x2170960, 0xc0000c0190, 0x2, 0x2, 0xc0002cbf50, 0x100742f, 0xc0000b4058, 0x0)
    /Users/henridf/go/pkg/mod/github.com/mccanne/charm@v0.0.3-0.20191224190439-b05e1b7b1be3/charm.go:77 +0x96
main.main()
    /Users/henridf/work/brim/zq/cmd/zq/main.go:9 +0x78
philrz commented 4 years ago

Verified in zq commit a14daef. The original repro steps now generate appropriate output rather than a crash.

$ zq alias.tzng | zq -t -
#myrec=record[host:ip]
#0:record[foo:myrec]
0:[[127.0.0.2;]]