search

segmentio / parquet-go

Go library to read/write Parquet files
https://pkg.go.dev/github.com/segmentio/parquet-go
Apache License 2.0
341 stars 102 forks source link

Enable *int64 to write as optional timestamp, *int32 to write as optional date. #472

Open DuanWeiFan opened 1 year ago

DuanWeiFan commented 1 year ago

At the moment, schema does not allow to be set from int32 to date, or int64 to timestamp.

We would like to add a statement that will redirect to it's Elem() whenever it encounters a reflect.Ptr.

This change was made in the past, but for some reason it was reverted later on. here Adding similar logic back and update schema_test.go

DuanWeiFan commented 1 year ago

Hi, can someone help take a look at this PR? The main purpose is to support writing pointers as Parquet schema.

DuanWeiFan commented 1 year ago

Hi @achille-roussel could you help take a look at this pr? At the moment parquet SchemaOf does not work for some pointers type. Would like to add this function so that it would work for all different pointers. Thank you.

achille-roussel commented 1 year ago

Hello @DuanWeiFan, I left Twilio so I'm currently not part of the maintainer team anymore for parquet-go.

Maybe someone else from the team can pick up the review? @kevinburkesegment @bartleyg