brimdata / super

A novel data lake based on super-structured data
https://zed.brimdata.io/
BSD 3-Clause "New" or "Revised" License
1.39k stars 64 forks source link

Mention Parquet among supported input formats #769

Closed philrz closed 4 years ago

philrz commented 4 years ago

zq as of commit 64d5e4a includes support for reading input files of Parquet format, but it's not yet mentioned in the supported list of formats in zq help.

$ zq help 2>&1 | grep "input data"
    -i format of input data [auto,zng,ndjson,zeek,zjson,tzng] (default "auto")

I checked with @aswan and he confirmed this was an oversight, so opening this as a reminder to circle back and add it.

philrz commented 4 years ago

Verified in zq commit a1d0967. Much like Prego spaghetti sauce in the 80's, it's in there.

$ zq help 2>&1 | grep "input data"
    -i format of input data [auto,zng,ndjson,zeek,zjson,tzng,parquet] (default "auto")
henridf commented 4 years ago

and in there!

$ zq -t  zio/detector/testdata/dns.parquet 
zio/detector/testdata/dns.parquet: format detection error
    tzng: line 1: bad format
    zeek: line 1: bad types/fields definition in zeek header
    ndjson: line 1: Unknown value type
    zjson: line 1: invalid character 'P' looking for beginning of value
    zng: zng descriptor out of range
    parquet: auto-detection not supported
aswan commented 4 years ago

If we're referencing 80's food commercials, we can't leave out https://www.youtube.com/watch?v=leQsRqMN230

philrz commented 4 years ago

Parkay truly shines when it speaks: https://www.youtube.com/watch?v=s0PnxGqFNz8