dbt-labs / dbt-utils

Utility functions for dbt projects.
https://hub.getdbt.com/dbt-labs/dbt_utils/latest/
Apache License 2.0
1.39k stars 498 forks source link

not_constant may generate false positive if used with group_by_column #848

Open igor-lobanov-maersk opened 1 year ago

igor-lobanov-maersk commented 1 year ago

Describe the bug

not_constant generic test would generate a false positive if used with group_by_column and there is only one row within a group

Steps to reproduce

Test model:

-- file test.sql
select 1 as a, 10 as b
union all
select 1 as a, 11 as b
union all
select 2 as a, 21 as b

Test schema.yml:

version: 2
models:
  - name: test
    columns:
      - name: b
        tests:
          - dbt_utils.not_constant:
              group_by_columns: ['a']

Run: dbt test -m test

Expected results

Test passes, because if there is only one value in the group, it does not break the constancy rule.

Actual results

Test fails.

Context

To explain what I'm doing and where this might be useful to others. I am flattening a nested array, and setting a flag if a given record is the first one. I wanted to add a test to make sure this logic always works, because I'm relying on this flag to be true exactly once for each id of the original record. In some cases there is only one element in the array, which gets the flag set to true correctly, but the test thinks it's wrong.

Screenshots and log output

Not sure what else would help.

System information

The contents of your packages.yml file:

packages:
  - package: dbt-labs/dbt_utils
    version: 1.1.1

Which database are you using dbt with? Dremio

The output of dbt --version:

Core:
  - installed: 1.5.8
  - latest:    1.6.6 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - dremio: 1.5.0 - Up to date!

Are you interested in contributing the fix?

PR provided #849

igor-lobanov-maersk commented 9 months ago

Hi team,

There has been no response to this issue report despite me providing a PR to fix it. I was wondering if this repo is still maintained?

github-actions[bot] commented 3 months ago

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

igor-lobanov-maersk commented 3 months ago

Hi @dbeatty10 and team -- I'm puzzled by complete lack of interest to this ticket... I didn't get a single comment either here on in the PR. Is this package being maintained at all?

dbeatty10 commented 3 months ago

Thanks for raising this PR @igor-lobanov-maersk !

Just now gave an initial review here: https://github.com/dbt-labs/dbt-utils/pull/849#pullrequestreview-2225726468