search

Tomme / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)
Apache License 2.0
140 stars 79 forks source link

Prepend "-- " to default `query_comment` #64

Closed daniel-cortez-stevenson closed 2 years ago

daniel-cortez-stevenson commented 2 years ago

Athena DDL does not always respect / ... / block quotations. This is the case with (at least) create external table ... statements.

Currently, if you try to run create external table ... statements, the following error is returned: pyathena.error.OperationalError: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'

Here I monkey-patched _QueryComment.add to prepend "-- " to the query_comment and replace any newlines with " ".

This allows the default query_comment to be added to create external table statements.

We might want to do this so that dbt-labs/dbt-external-tables can add Athena support as is proposed in dbt-labs/dbt-external-tables/pull/133

Without this modification, the only way to run dbt run-operation stage_external_sources --vars "ext_full_refresh: true" without an error is to set query_comment: to nothing, a specific value, or a macro.

daniel-cortez-stevenson commented 2 years ago

Hey @Tomme - any interest in pulling this in?

Gatsby-Lee commented 2 years ago

@Tomme @divy9393 What is the status of this PR? If this change looks good, can you merge this into the master?

Gatsby-Lee commented 2 years ago

@Tomme Thank you

Tomme commented 2 years ago

@Tomme Thank you

No worries at all - apologies for the delay!