search

brimdata / super

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.39k stars 64 forks source link

tzng parser panics on malformed record #638

Closed henridf closed 4 years ago

henridf commented 4 years ago 
10:45 ~/work/brim/zq(master)
$ git rev-parse --short HEAD
6a05953
10:45 ~/work/brim/zq(master)
$ cat blar.zng
#0:record[blar:string]
0:[blar1]
10:45 ~/work/brim/zq(master)
$ zq blar.zng
panic: runtime error: slice bounds out of range [7:6]
philrz commented 4 years ago

Verified in zq commit a4931cd. We no longer panic but instead return helpful error messages for all formats we attempted to auto-detect.

# cat blar.tzng 
#0:record[blar:string]
0:[blar1]

# zq -t blar.tzng 
blar.tzng: format detection error
    tzng: line 2: zng syntax error: unterminated container
    zeek: line 2: bad types/fields definition in zeek header
    ndjson: line 1: Unknown value type
    zjson: line 1: invalid character '#' looking for beginning of value
    zng: malformed zng record

Thanks @henridf!