Closed sanga8 closed 2 months ago
When running dbt seed the table gets loaded properly the first time but then it fails on a second run, despite setting the seed_mode to overwrite.
dbt seed
LOGS if (spark.sql("show tables in MYDATABASE").where("tableName == 'MYTABLE'").count() > 0): df.write .mode("overwrite") .format("parquet") .insertInto(table_name, overwrite=True) else: df.write .option("path", "http://s3://my-table-path") .format("parquet") .saveAsTable(table_name) SqlWrapper2.execute("""select * from MYDATABASE.MYTABLE limit 1""") , AnalysisException: Table MYDATABASE.MYTABLE already exists.
VERSION dbt-glue==1.7.2
When running
dbt seedthe table gets loaded properly the first time but then it fails on a second run, despite setting the seed_mode to overwrite.LOGS if (spark.sql("show tables in MYDATABASE").where("tableName == 'MYTABLE'").count() > 0): df.write .mode("overwrite") .format("parquet") .insertInto(table_name, overwrite=True) else: df.write .option("path", "http://s3://my-table-path") .format("parquet") .saveAsTable(table_name) SqlWrapper2.execute("""select * from MYDATABASE.MYTABLE limit 1""") , AnalysisException: Table MYDATABASE.MYTABLE already exists.
VERSION dbt-glue==1.7.2