Unable to write Timestamp Logical Type

aloneguid / parquet-dotnet

Fully managed Apache Parquet implementation

https://aloneguid.github.io/parquet-dotnet/

MIT License

542 stars 140 forks source link

Unable to write Timestamp Logical Type #492

Open kharlm16 opened 3 months ago

kharlm16 commented 3 months ago

Issue description

Issue

Writing DateTimes using DateTimeDataField uses TIMESTAMP_MILLIS Converted Type instead of TIMESTAMP(isAdjusted, unit) Logical Type. Is this by design or are there any plans to support writing TIMESTAMP logical type?

aloneguid commented 2 months ago

I have no plans to support this at the moment, so happy for you to pick it up.

kja-xyt commented 8 hours ago

It would be great to have it (NANOS to be specific). Files written with Apache Arrow have something like that in Thrift Metadata:

      {
        "Path": "MyNanoTime",
        "Type": "INT64",
        "TypeLength": null,
        "LogicalType": {
          "Name": "TIME",
          "IsAdjustedToUTC": false,
          "Unit": "NANOS"
        },
        "RepetitionType": "REQUIRED",
        "ConvertedType": "",
        "Children": []
      },

and you can read them in Python without the need to cast anything.