search

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.49k stars 1.68k forks source link

[Feature Request] #2163

Open alxvsk opened 11 months ago

alxvsk commented 11 months ago

Feature request

Which Delta project/connector is this regarding?

Overview

Ability to set time travel option in Hive Metastore catalog for HMS-managed Delta Lake tables.

Motivation

Need to provide time travel options from the metadata catalog to deliver users a historical view of the table if the time travel option is not explicitly specified in Spark. For example, Hive Metastore / Catalog will provide either versionAsOf or timestampAsOf SerDe parameters (similar to path, which is already stored in SerDe parameter in table’s Storage Descriptor to specify location of Delta table) to achieve this goal.

Further details

Time travel options from the Hive Metastore catalog will work the same as if it was set from Delta format options, only one versionAsOf or timestampAsOf is allowed.

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

Note: already have a patch for Spark available and do not have anything for Trino.

tdas commented 11 months ago

Delta Trino connector is not in this repository. It is in the Trino repository - https://github.com/trinodb/trino/tree/master/plugin/trino-delta-lake

So its best to create issues to improve the Delta Trino connector in that repository.