search

Tomme / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
Apache License 2.0
142 stars 79 forks source link

Seeds fail to update when already applied #113

Open Nintorac opened 2 years ago

Nintorac commented 2 years ago

If seeds have already been then subsequent runs fail with a unhelpful error

The error that DBT shows is

14:51:05  1 of 4 START seed file dbt_test.cohorts ........................................ [RUN]
14:51:06  1 of 4 ERROR loading seed file dbt_test.cohorts ................................ [ERROR in 0.28s]
14:51:06  2 of 4 START seed file dbt_test.diagnosis ...................................... [RUN]
14:51:06  2 of 4 ERROR loading seed file dbt_test.diagnosis .............................. [ERROR in 0.15s]
14:51:06  3 of 4 START seed file dbt_test.parameters ..................................... [RUN]
14:51:06  3 of 4 ERROR loading seed file dbt_test.parameters ............................. [ERROR in 0.13s]
14:51:06  4 of 4 START seed file dbt_test.valid_ranges ................................... [RUN]
14:51:06  4 of 4 ERROR loading seed file dbt_test.valid_ranges ........................... [ERROR in 0.13s]
14:51:06  
14:51:06  Finished running 4 seeds in 12.70s.
14:51:06  
14:51:06  Completed with 4 errors and 0 warnings:
14:51:06  
14:51:06  Runtime Error in seed cohorts (seeds/meta/cohorts.csv)
14:51:06    local variable 'table' referenced before assignment
14:51:06  
14:51:06  Runtime Error in seed diagnosis (seeds/meta/diagnosis.csv)
14:51:06    local variable 'table' referenced before assignment
14:51:06  
14:51:06  Runtime Error in seed parameters (seeds/meta/parameters.csv)
14:51:06    local variable 'table' referenced before assignment
14:51:06  
14:51:06  Runtime Error in seed valid_ranges (seeds/meta/valid_ranges.csv)
14:51:06    local variable 'table' referenced before assignment

and if you dig into the logs you can find 

13:42:45.927212 [info ] [Thread-1  ]: 1 of 4 START seed file dbt.cohorts ............................................. [RUN]
13:42:45.927659 [debug] [Thread-1  ]: Acquiring new athena connection "seed.emr.cohorts"
13:42:45.928175 [debug] [Thread-1  ]: Began compiling node seed.emr.cohorts
13:42:45.928367 [debug] [Thread-1  ]: finished collecting timing info
13:42:45.928537 [debug] [Thread-1  ]: Began executing node seed.emr.cohorts
13:42:45.959518 [debug] [Thread-1  ]: Opening a new connection, currently in state closed
13:42:46.214376 [debug] [Thread-1  ]: Athena adapter: Error running SQL: macro drop_relation
13:42:46.214740 [debug] [Thread-1  ]: finished collecting timing info
13:42:46.214926 [debug] [Thread-1  ]: On seed.emr.cohorts: Close
13:42:46.215248 [debug] [Thread-1  ]: Runtime Error in seed cohorts (seeds/meta/cohorts.csv)
  local variable 'table' referenced before assignment

The issue as far as I can tell is this line Athena adapter: Error running SQL: macro drop_relation

VDFaller commented 2 years ago

This is the same root problem I was having in #107 . It looks like #74 fixes it. My guess is you're using a profile_name and not environment variables?

Nintorac commented 2 years ago

My guess is you're using a profile_name and not environment variables?

correct

nice, hopefully that lands soon then :)