The bug occurs when you set up custom_location for the incremental model with default table_format (Hive). Initial run of the incremental model won`t cause any issues since tmp table is only created when the table exists. However, the second run will lead to the creation of the _tmp table in the same S3 path where the original incremental table is stored. This will cause two tables to be stored in the same location.
Steps To Reproduce
1) Create an incremental model without any table format. Specify custom_location property
2) Create your incremental table using dbt run command
3) Run the same model a second time and track where the tmp table is created
Actual behavior
Temp table is created in the same S3 location where original table is created
Describe the bug
The bug occurs when you set up custom_location for the incremental model with default table_format (Hive). Initial run of the incremental model won`t cause any issues since tmp table is only created when the table exists. However, the second run will lead to the creation of the _tmp table in the same S3 path where the original incremental table is stored. This will cause two tables to be stored in the same location.
Steps To Reproduce
1) Create an incremental model without any table format. Specify custom_location property 2) Create your incremental table using dbt run command 3) Run the same model a second time and track where the tmp table is created
Actual behavior
Temp table is created in the same S3 location where original table is created
Expected behavior
Temp table has its own location
System information
https://github.com/aws-samples/dbt-glue/blob/ec464541c982503730c2a81f804ab9b542a2548e/dbt/include/glue/macros/adapters.sql#L87-L98 https://github.com/aws-samples/dbt-glue/blob/ec464541c982503730c2a81f804ab9b542a2548e/dbt/include/glue/macros/adapters.sql#L1-L15