Append Only GreemPlum tables can take up a lot of space on disks that is actually not being utilized. Sometimes it is required to create heap based tables for dbt-models.
Desired behavior
{{
config(
materialized='table',
appendonly=false,
schema='test',
)
}}
select
floor(random()*(1000))+10 as id,
current_timestamp as ts
In order for example above to work and create not Append Only table it should produce following SQL statement:
create table "test"."test_datamart_tmp"
with (
appendonly=False
)
as (
select
floor(random()*(1000))+10 as id,
current_timestamp as ts
)
DISTRIBUTED RANDOMLY
;
SQL statement should not cotain any of blocksize, orientation, compresstype, compresslevel parameters otherwise GreenPlum produces ERROR: invalid option "orientation" for base relation Hint: Table orientation only valid for Append Only relations, create an AO relation to use table orientation.
Current behavior
Currently example above produces following SQL statement:
create table "test"."test_datamart__dbt_tmp"
with (
appendonly=False,
blocksize=32768,
orientation=column,
compresstype=NONE,
compresslevel=4
)
as (
select
floor(random()*(1000))+10 as id,
current_timestamp as ts
)
DISTRIBUTED RANDOMLY
;
Possible solution:
Check for parameter appendonly when generating SQL statements to run and if it is set to false do not include blocksize, orientation, compresstype, compresslevel parameters in SQL statement.
Append Only GreemPlum tables can take up a lot of space on disks that is actually not being utilized. Sometimes it is required to create heap based tables for dbt-models. Desired behavior
In order for example above to work and create not Append Only table it should produce following SQL statement:
SQL statement should not cotain any of
blocksize
,orientation
,compresstype
,compresslevel
parameters otherwise GreenPlum produces ERROR: invalid option "orientation" for base relation Hint: Table orientation only valid for Append Only relations, create an AO relation to use table orientation.Current behavior Currently example above produces following SQL statement:
Possible solution: Check for parameter appendonly when generating SQL statements to run and if it is set to false do not include
blocksize
,orientation
,compresstype
,compresslevel
parameters in SQL statement.Context: dbt-core 1.2.2 dbt-greenplum 1.2.0