Open aPeterHeise opened 1 year ago
Note, the way that #138 is implemented, option 1 via "external tables" should already work.
- name: iceberg_tbl
description: "External table using Iceberg files"
external:
location: "@stage" # reference an existing external stage
file_format: "( type = parquet )" # fully specified here, or reference an existing file format
table_format: iceberg # specify the table format
auto_refresh: false # requires configuring an event notification from Amazon S3 or Azure
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.
table_format: iceberg
table_format: iceberg
is not supported for create or replace external table anymore.
@ShahNewazKhan are you suggesting that TABLE_FORMAT = ICEBERG
is not possible in Snowflake's CREATE EXTERNAL TABLE
command? therefore not possible to add support for it in dbt-external-tables?
@dataders correct, if you try to use CREATE EXTERNAL TABLE
with TABLE_FORMAT = ICEBERG
you get the following error:
You need to use CREATE OR REPLACE ICEBERG TABLE and specify an external_volume
(similar to a snowflake named stage
).
For creating the external iceberg table we would need something like:
CREATE OR REPLACE ICEBERG TABLE {database}.{schema}.{table}
EXTERNAL_VOLUME='{external_volume}'
CATALOG='{glue_catalog_integration}'
CATALOG_TABLE_NAME='{glue_table}'
CATALOG_NAMESPACE='{glue_database}';
The existing code doesn't support it.
Snowflake added support for Apache Iceberg.
There seem to be two ways on how this will be implemented:
While this feature is still in private preview, I'd like to add it to the backlog here already.