search

aloneguid / parquet-dotnet

Fully managed Apache Parquet implementation
https://aloneguid.github.io/parquet-dotnet/
MIT License
605 stars 151 forks source link

Field name with dot (.) #82

Closed felipepessoto closed 3 years ago

felipepessoto commented 4 years ago

This commit https://github.com/aloneguid/parquet-dotnet/commit/16b55d54bc7f096eefb9df8227b1f321d9839ee5 added a validation to disallow fields with dots.

Do we really have a reason to don't accept dots? If I comment the throw line I can read a file with dots in column names, and the only failing test is the test that validates if it throws when we use dots.

I couldn't find any spec saying dots are not allowed.

Thanks.

kirchsth commented 4 years ago

A dot is the Schema.PathSeparator and used in e.g. paths of StructField's as separator if you store e.g. (list of) structs

felipepessoto commented 3 years ago

The problem is, some viewer like ParquetFileViewer can't open the parquet files if column has a dot.

felipepessoto commented 3 years ago

Any update? Is this by design? Is it possible to fix it, or have a opt-in option to allow dots?

aloneguid commented 3 years ago

Parquet spec does allow dots so Id say we should too.

felipepessoto commented 3 years ago

Fixed