segmentio / parquet-go

Go library to read/write Parquet files
https://pkg.go.dev/github.com/segmentio/parquet-go
Apache License 2.0
341 stars 102 forks source link

Increment truncated binary in byteArrayColumnIndexer #496

Closed ty-sentio-xyz closed 1 year ago

ty-sentio-xyz commented 1 year ago

Byte array will be increased by 1 after truncated, for both byteArrayColumnIndexer and fixedLenByteArrayColumnIndexer.

This is to make sure truncated values are lexicographically correct, when comparing truncated value with actual max value.

ty-sentio-xyz commented 1 year ago

Thanks for the comments.

I've updated the PR with a new test case that covers the case.

Not sure this is the right way of doing it but I am separating these tests into two files:

The new test case is: