Open parsnips opened 1 year ago
Failing test here https://github.com/parsnips/parquet-go/commit/7aff1dac1e6cd5037d352705847d9b0bb2369aef
I've stepped through with a debugger a number of times and it looks like reconstructFuncOfMap
is never called even though it's setup on the schema.
When hitting those map columns it seems to always get set to zero value:
Note this doesn't work with parquet with the (MAP)
annotations either:
optional group new (MAP) {
repeated group map (MAP_KEY_VALUE) {
required binary key (UTF8);
optional group value {
optional binary b (UTF8);
optional binary n (UTF8);
}
}
}
I have a parquet file that i used the
PrintSchema
on and one of the maps it prints the following schema snippet:Which appears to be a deprecated form of a map (outputted from AWS kinesis firehose parquet transformer).... I tried to model this using the following
HasMap
:However when I read the rows from my parquet file, the golang maps are empty. Printing the schema for the struct, I noticed it was significantly different:
Any advice to build structs that work with
repeated group map
?