[Feature Request][Standalone] `TimestampNTZ` datatype is not supported

JaySardhara commented 5 months ago

Feature request

Which Delta project/connector is this regarding?

[ ] Spark
[x] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Overview

Delta standalone seems to not TimestampNTZ datatype supported. Only Timestamp is available but as per Spark and Delta documents, TimestampNTZ should be supported as well since other connectors and delta-spark has support for it.

Motivation

TimestampNTZ is necessary since there is no way to create tables with delta-standalone where user can create a column of timestamp without timezone info.

Further details

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

[ ] Yes. I can contribute this feature independently.
[x] Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
[ ] No. I cannot contribute this feature at this time.

tdas commented 5 months ago

We have added support for timestamp NTZ in Delta Kernel project - https://github.com/delta-io/delta/tree/master/kernel Delta Kernel is going to be the replacement of Delta Standalone as it will have much better capabilities that Standalone. If you want to learn more about why... see my talk - https://www.youtube.com/watch?v=KVUMFv7470I (sorry for the shameless plug).

So we are not going to add any new functionality to Standalone and encourage you to migrate to Kernel. We are happy to help in that process and learn from it.

cc @vkorukanti can you confirmed which version of kernel has / will have timestamp ntz support?

vkorukanti commented 5 months ago

@JaySardhara Timestamp_ntz support in Kernel reads will be part of the 3.2 which is targetted for release on May 7, 2024. Here is the ticket.

delta-io / delta