Closed etheleon closed 1 year ago
see discussion https://github.com/dbt-labs/dbt-spark/discussions/847
@fleid: Could you merge this PR?
@colin-rogers-dbt can you review this? :bow:
@JCZuurmond are we doing this one? Looks like the changelog entry is missing.
@etheleon can you add a changelog entry?
hey @colin-rogers-dbt working on it 👍
SHOW CREATE TABLE
to get the table definition now it includes tblproperties
/*
Welcome to your first dbt model!
Did you know that you can also configure models directly within SQL files?
This will override configurations stated in dbt_project.yml
Try changing "table" to "view" below
*/
{{
config(
location_root='s3a://some_bucket/tables',
tblproperties = {
'format': 'iceberg/parquet',
'format-version': '2',
'write.delete.mode': 'merge-on-read',
'write.delete.target-file-size-bytes': '536870912',
'write.merge.mode': 'merge-on-read',
'write.metadata.delete-after-commit.enabled': 'true',
'write.metadata.previous-versions-max': '100',
'write.target-file-size-bytes': '536870912',
'write.update.mode': 'merge-on-read'}
)
}}
with source_data as (
select 1 as id
union all
select null as id
)
select *
from source_data
/*
Uncomment the line below to remove records with null `id` values
*/
-- where id is not null
output from SHOW CREATE TABLE
CREATE TABLE Iceberg catalog.ds.my_first_dbt_model ( id INT) USING iceberg LOCATION 's3a://some_bucket/tables/my_first_dbt_model' TBLPROPERTIES ( 'current-snapshot-id' = '1344159441507971776', 'format' = 'iceberg/parquet', 'format-version' = '2', 'write.delete.mode' = 'merge-on-read', 'write.delete.target-file-size-bytes' = '536870912', 'write.merge.mode' = 'merge-on-read', 'write.metadata.delete-after-commit.enabled' = 'true', 'write.metadata.previous-versions-max' = '100', 'write.target-file-size-bytes' = '536870912', 'write.update.mode' = 'merge-on-read')
resolves #865 resolves #190 docs dbt-labs/docs.getdbt.com/#
Problem
tblproperties
clause macro is not used when creating tables. Add macro to adaptor.sqlSolution
Add macro
Checklist