search

apache / iceberg

Apache Iceberg
https://iceberg.apache.org/
Apache License 2.0
6.42k stars 2.22k forks source link

Iceberg to Redshift load #6841

Closed nia94 closed 1 month ago

nia94 commented 1 year ago

Query engine

No response

Question

Has anyone tried loading data from iceberg tables to Redshift DB? Any suggestions are much appreciated. We have a use case to load data from iceberg tables to Redshift

toddy86 commented 1 year ago

As at today, Redshift Spectrum can't directly read Iceberg tables (which is a massive shame considering Athena can). AFAIK, the only real way to get your data from Iceberg and into Redshift would be to use Athena (or PySpark or insert-iceberg-compatible-engine-here) to unload the tables or partitions you need into vanilla Parquet, then load the data into Redshift. Hopefully AWS updates Redshift to read from Iceberg tables shortly (they already allow reads from Delta tables)

alvsanand commented 1 year ago

Iceberg via Spectrum is in preview as of today https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-redshift-querying-apache-iceberg-tables/

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] commented 1 month ago

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'