search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.37k stars 1.69k forks source link

SQL Time Travel on Iceberg Tables #24937

Open chrisfw opened 1 year ago

chrisfw commented 1 year ago

Feature request

Is your feature request related to a problem? Please describe. Time travel SQL Syntax is not supported in Iceberg Connector

Describe the solution you'd like

Support the Spark SQL syntax for time travel queries in Iceberg via the StarRocks Iceberg Connector

-- time travel to October 26, 1986 at 01:21:00
SELECT * FROM prod.db.table TIMESTAMP AS OF '1986-10-26 01:21:00';

-- time travel to snapshot with id 10963874102873L
SELECT * FROM prod.db.table VERSION AS OF 10963874102873;

-- time travel to the head snapshot of audit-branch
SELECT * FROM prod.db.table VERSION AS OF 'audit-branch';

-- time travel to the snapshot referenced by the tag historical-snapshot
SELECT * FROM prod.db.table VERSION AS OF 'historical-snapshot';

Describe alternatives you've considered

This feature does not really have an alternative/workaround other than the MySQL Connector where time travel queries were implemented for Dolt by a Dolt Engineer Additional context Reference: Iceberg Spark Queries

imay commented 1 year ago

@chrisfw Have you already used StarRocks to query Apache Iceberg?

chrisfw commented 1 year ago

@imay , I have been working with @alberttwong to get the Iceberg query functionality working from my StarRocks installation as we have hit some hurdles. My understanding is that the time travel functionality has not yet been implemented regardless.

github-actions[bot] commented 7 months ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!

chrisfw commented 7 months ago

Hello,

I see that time-travel queries are on the 2023 roadmap so I am just wondering if there is any idea at this point of what version the functionality may be included in?

Regards, Chris

inviscid commented 6 months ago

Hi - any update on whether this is scheduled for a future release? Having the ability to use native StarRocks storage for optimal performance while also enabling time travel (with a little slower performance) on specific tables is an incredible combination of options.

github-actions[bot] commented 1 week ago

We have marked this issue as stale because it has been inactive for 6 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to StarRocks!

chrisfw commented 1 week ago

I am just wondering if this feature is anywhere on the 2024 roadmap?

mrhamburg commented 1 week ago

I am just wondering if this feature is anywhere on the 2024 roadmap?

https://github.com/StarRocks/starrocks/pull/47997 and https://github.com/StarRocks/starrocks/issues/47846

chrisfw commented 1 week ago

@mrhamburg , thank you very much for the information. That is great news!

Regards, Chris