dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.25k stars 1.53k forks source link

[Bug] Redshift sql_header, doesn't work, but doesn't error either #9819

Open ann8ty opened 3 months ago

ann8ty commented 3 months ago

Is this a new bug in dbt-core?

Current Behavior

Per doc https://docs.getdbt.com/reference/resource-configs/sql_header I've tried both:

models:
  +sql_header: "SET enable_case_sensitive_super_attribute to TRUE;"
{{ config(
    sql_header = "SET enable_case_sensitive_super_attribute to TRUE;")}}

we've also tried ON instead of TRUE

and the result is the column value is NULL when run through DBT, but the same query run through redshift query editor produces a value, leading us to believe that enable_case_sensitive_super_attribute isn't being set by DBT.

this is a per session attribute in redshift, unable to set as a parameter group

the rest of the dbt staging sql

with i_json as (
SELECT
    JSON_PARSE(inv) as i_super,
FROM {{ source('dynamodb_raw', 'table_status' )}} t )
select
    i."isNet" as is_net,
from i_json t, t.i_super i

Expected Behavior

expected DBT to work the same as redshift query editor

Steps To Reproduce

  1. dbt run
  2. look at output, its null
  3. same query in redshift query editor, has value

Relevant log output

"completed successfully"

Environment

- OS: mac os sonoma 14.4 (23E214)
- Python: Python 3.12.2
- dbt: 1.4.7

Which database adapter are you using with dbt?

redshift

Additional Context

is it possibly not sql_header for redshift? there is no example for redshift.

graciegoheen commented 3 months ago

Hi! Trying to figure out what might be going on here. You mentioned "this is a per session attribute in redshift". I wonder if we're sending the statements to the database in a way that each statement becomes its own session. Would you be able to try it on 1.5, which uses redshift-connector instead of psycopg2 and see if the behavior is any different?

github-actions[bot] commented 5 days ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.