Tomme / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
Apache License 2.0
140 stars 79 forks source link

Source with different database than target database not working #89

Open nghildebrandt opened 2 years ago

nghildebrandt commented 2 years ago

Hi, I am wondering if the database property for sources is defined for dbt athena? After setting the database property in sources and running dbt run, it tries to build tables using my target database and not source database. I hope you can help with this, I have attached my sources file. Thanks

To select from the source I call: from {{source('fleetdog', 'raw_public_trips')}}

My source is configured like this:

version:  2

sources:
  - name: fleetdog
    database: dataalpha
    quoting:
      database: true
    tables:
      - name: raw_public_trips

Error:

07:41:39  Runtime Error in model my_first_dbt_model (models/example/my_first_dbt_model.sql)
07:41:39    SYNTAX_ERROR: line 15:6: Table awsdatacatalog.fleetdog.raw_public_trips does not exist. 
prabavnc commented 2 years ago

Please change it to

schema: dataalpha

It should work

nghildebrandt commented 2 years ago

Hi so this still doesn't work. In the downstream model, I want it to be compiled to:

dataalpha.fleetdog.raw_public_trips

privatedumbo commented 1 year ago

Hi so this still doesn't work. In the downstream model, I want it to be compiled to:

dataalpha.fleetdog.raw_public_trips

I had to do this for a source. I ended up setting, in the schema field, the . . That compiled the query to the full length name and worked

arturzangiev commented 1 year ago

Doesn't work for me. I also have my main database is AwsDataCatalog, but would like to specify database as mysql. So when it compiles it would be good to have a structure AwsDataCatalog.SCHEMA.TABLE and in the other case mysql.SCHEMA.TABLE. But in my case it appears as SCHEMA.TABLE

jussapaavo commented 1 year ago

I also have the same issue. For me, it does not compile the database: dbname into the actual query, it just outputs schema.table without the database specification in front.

I used a workaround with configurating the schema as schema: dbname.schema which then compiles to dbname.schema.table in the query.