Dandandan / dbt-athena

dbt adapter for Athena
Apache License 2.0
39 stars 9 forks source link

Catalog does not exist #17

Closed asyarif93 closed 4 years ago

asyarif93 commented 4 years ago

Hi, Thanks for the great work on this adapter!

Somehow when i trying to run command dbt run I get

Running with dbt=0.15.1
Found 1 model, 0 tests, 0 snapshots, 0 analyses, 128 macros, 0 operations, 0 seed files, 0 sources

Encountered an error:
Runtime Error
  SYNTAX_ERROR: line 2:10: Catalog testing does not exist

here is the dbt_project.yml

name: 'test_athena'
version: '0.0.1'
profile: 'athena'
source-paths: ["models"]
analysis-paths: ["analysis"] 
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]

and this is profiles.yml

athena:
  outputs:
    dev:
      type: athena
      threads: 1
      database: testing
      region_name: ap-southeast-1
      schema: testing
      s3_staging_dir: s3://test_staging_dbt
  target: dev

and this is dbt-debug results

dbt debug
Running with dbt=0.15.1
dbt version: 0.15.1
python version: 3.7.4
python path: /home/user/python/dbt_impl/bin/python3
os info: Linux-4.15.0-88-generic-x86_64-with-debian-buster-sid
Using profiles.yml file at /home/user/.dbt/profiles.yml

Configuration:
  profiles.yml file [OK found and valid]
  dbt_project.yml file [OK found and valid]
  profile: athena [OK found]
  target: dev [OK found]

Required dependencies:
 - git [OK found]

Connection:
  s3_staging_dir: s3://test_staging_dbt
  database: testing
  schema: testing
  region_name: ap-southeast-1
  Connection test: OK connection ok

As far as i know, catalog represent schema_name in PyAthena. I assume this error came from athena query rather than dbt it self. so i tried to create connection in PyAthena and run same queries. it run perfectly. do you have some clues?

Thanks again!

asyarif93 commented 4 years ago

i've found that dbt tries to run

select distinct schema_name
    from testing.INFORMATION_SCHEMA.schemata

which give result SYNTAX_ERROR: line 2:10: Catalog testing does not exist as the query should be

select distinct schema_name
    from INFORMATION_SCHEMA.schemata
Dandandan commented 4 years ago

Can you try to use as database awscatalog in your profile.yml?

asyarif93 commented 4 years ago

it also fail giving result:

Runtime Error
  SYNTAX_ERROR: line 2:10: Catalog awscatalog does not exist

but when i trying to use awsdatacatalog in profile.yml dbt runs perfectly