calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros
https://calogica.github.io/dbt-expectations/
Apache License 2.0
1.04k stars 126 forks source link

expect_select_column_values_to_be_unique_within_record not consistent with dbt built-in test unique #183

Closed greghor closed 2 years ago

greghor commented 2 years ago

hi,

I noticed that the dbt-expectations test expect_select_column_values_to_be_unique_within_record passes on some of my table while the built-in dbt test unique fails.

image

After checking the value, the column id_version does have duplicates, which makes me think that expect_select_column_values_to_be_unique_within_record doesn't behave properly.

Here is how I use the dbt-expectations test

models:
  - name: my_table
    tests:
      - dbt_expectations.expect_select_column_values_to_be_unique_within_record:
          column_list: [id_version]

Versions:

dbt-expectations: 0.5.8
dbt-core: 1.2.1
dbt-bigquery: 1.2.0 
clausherther commented 2 years ago

Hi @greghor - I think you want to use expect_column_values_to_be_unique (https://github.com/calogica/dbt-expectations#expect_column_values_to_be_unique) instead. The test expect_select_column_values_to_be_unique_within_record is a bit more specialized, it tests that within a row, the values across two or more columns are unique. So, in your case this will always be true since you're only testing one column. It's a confusing test, but part of the Great Expectations package, so included here, and we could probably do better on documentation. Thanks for flagging this.

greghor commented 2 years ago

Oh I see - my mistake then. Thanks for making this clear!