search

xitongsys / parquet-go

pure golang library for reading/writing parquet file
Apache License 2.0
1.27k stars 293 forks source link

DELTA_BINARY_PACKED accepted for invalid types DOUBLE and FLOAT #352

Closed damiendr closed 3 years ago

damiendr commented 3 years ago

parquet-go will happily accept DELTA_BINARY_PACKED for floating point numbers. As far as I understand this is not allowed by the Parquet spec.

At the moment parquet-go does not raise any error and creates a .parquet file that cannot be read back by parquet-tools:

Go source: https://gist.github.com/damiendr/60b67d9861f5cce3479861b82f7e9458

$ go build
$ ./test-parquet
$ parquet-tools cat --json out.parquet
org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file file:/path/out.parquet
xitongsys commented 3 years ago

hi, @damiendr Fixed in this merge from @pkf (https://github.com/xitongsys/parquet-go/pull/359). Thanks @pkf