aws-samples / dbt-glue

This repository contains the dbt-glue adapter
Apache License 2.0
101 stars 69 forks source link

Fix unraised DbtDatabaseError #434

Closed Jeremynadal33 closed 2 months ago

Jeremynadal33 commented 2 months ago

resolves #433

Description

Modified the DbtDatabaseError from dbt to dbt_common

Checklist

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

FYI : I am not sure wether this change needs testing, if it does, I would be interested in understanding how to!

Tests that were run locally

Same as in the #433 issue but then got the following output :

09:19:04  Running with dbt=1.8.4
09:19:05  Registered adapter: glue=1.8.1
09:19:05  Found 41 models, 2 snapshots, 1 seed, 2 operations, 1 test, 1 source, 1367 macros
09:19:05  
09:19:07  
09:19:07  
09:19:07  1 of 1 START sql incremental model xxx.test_database_error ............ [RUN]
09:19:17  Glue adapter: Glue returned `error` for statement None for code 

spark.sql("""

select "a" as id
""").write.format("delta").mode("overwrite").save("s3://xxx/xxx/test_database_error")
SqlWrapper2.execute("""select 1""")
, AnalysisException: Failed to merge fields 'id' and 'id'. Failed to merge incompatible data types IntegerType and StringType
09:19:17  1 of 1 ERROR creating sql incremental model xxx.test_database_error ... [ERROR in 10.82s]
09:19:35  
09:19:35  
09:19:35  Finished running 1 incremental model, 2 project hooks in 0 hours 0 minutes and 29.55 seconds (29.55s).
09:19:35  
09:19:35  Completed with 1 error and 0 warnings:
09:19:35  
09:19:35    Database Error in model test_database_error (src/models/test_database_error.sql)
  GlueDeltaWriteTableFailed
09:19:35  
09:19:35  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

The error is well raised

moomindani commented 2 months ago

Thanks for your contribution!