search

Tomme / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
Apache License 2.0
140 stars 79 forks source link

prehook sql throw exception #129

Open lichenglin opened 1 year ago

lichenglin commented 1 year ago

I wanted to excute a pre_hook query like this {{ config(materialized='view',pre_hook="drop table if exists my_first_dbt_model") }} But I found that in athena , the sql become -- /* {"app": "dbt", "dbt_version": "1.2.1", "profile_name": "local", "target_name": "dev", "node_id": "model.dbtest.my_first_dbt_model"} */ drop table if exists my_first_dbt_model since all the sql stay in one line , the sql had been miss as a comment , It failed

hagai-arad commented 1 year ago

@daniel-cortez-stevenson Hi, can this be related to https://github.com/Tomme/dbt-athena/pull/64? If so, do you know how to fix it?

Thanks!

daniel-cortez-stevenson commented 1 year ago

Yea must be - we can fix that

nicor88 commented 1 year ago

Removing -- from the comment doesn't fix all the cases, e.g. alter statement:

/* {"app": "dbt", "dbt_version": "1.3.0", "profile_name": "athena", "target_name": "dev", "node_id": "model.lakehouse.example_iceberg_increment"} */ ALTER TABLE silver.example_iceberg_increment SET TBLPROPERTIES ('vacuum_max_snapshot_age_seconds'='86400')
daniel-cortez-stevenson commented 1 year ago

Appreciate the comment @nicor88 - after seeing this I thought that statements starting on the same line as the multi-line comment ends would be executed.

I wouldn't remove the --, I'd add a newline after the annotation.

Do we agree that's a sufficient/wanted fix?

nicor88 commented 1 year ago

@daniel-cortez-stevenson feel free to open a PR here: https://github.com/dbt-athena/dbt-athena

The specific issue that I was having was cover by this pr: https://github.com/dbt-athena/dbt-athena/pull/56 as somehow I thought as well to remove -- and add a new line \n was still leading to issue.

Anyhow, the development of Tomme/dbt-athena seems a bit stuck, the community did quite some progress on the new fork, have a look.