dbt-labs / dbt-audit-helper

Useful macros when performing data audits
https://hub.getdbt.com/dbt-labs/audit_helper/latest/
Apache License 2.0
302 stars 37 forks source link

compare_column_values not working with Athena/Presto #87

Open evabilski opened 7 months ago

evabilski commented 7 months ago

Describe the bug

Function compare_column_values not working on Athena/Presto.

Steps to reproduce

Create test in Presto environment:

{% set old_etl_relation_query %}
    select
        'a' AS col_a,
        'b' AS col_b
{% endset %}

{% set new_etl_relation_query %}
    select
        'a' AS col_a,
        'b' AS col_b
{% endset %}

{{ audit_helper.compare_column_values(
    a_query=old_etl_relation_query,
    b_query=new_etl_relation_query,
    primary_key="col_a",
    column_to_compare="col_b"
)
}}

Log output

COLUMN_NOT_FOUND: line 59:10: Column 'column_name' cannot be resolved or requester is not authorized to access requested resources

System information

dbt-labs/audit_helper version: 0.9.0

Which database are you using dbt with?

Additional context

Presto does not support grouping by aliases. Line 53 of compare_column_values.sql: group by column_name should be removed in order to work with Presto

Are you interested in contributing the fix?

I can try making a PR

sanga8 commented 2 months ago

I still have this issue in version 0.11.0