calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros
https://calogica.github.io/dbt-expectations/
Apache License 2.0
1.05k stars 127 forks source link

[BUG] Issue in expect_column_values_to_be_of_type for field under source #314

Open dominikszewczyk opened 2 months ago

dominikszewczyk commented 2 months ago

Is this a new bug in dbt-expectations?

Current Behavior

Using the dbt_expectations.expect_column_values_to_be_of_type test on a FLOAT field in source return following error: "'str' object has no attribute 'database'". In dbt.log file I see that this error comes from _File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\adapters\bigquery\impl.py", line 226, in get_columns_inrelation database=relation.database, schema=relation.schema, identifier=relation.identifier

Expected Behavior

Test should return Error when data type is not float or should be missed when all values are float.

Steps To Reproduce

source.yaml

sources:
  - name: workspace
    database: db_name
    schema: workspace
    tables:
      - name: raw_mapping_vat
        columns:
          - name: vat_ratio
            description: ""
            tests:
              - dbt_expectations.expect_column_values_to_be_of_type:
                  column_type: float
                  # config:
                  #   severity: warn
                  #   warn_if: "!=0"

Relevant log output

Terminal:

Unhandled error while executing 
'str' object has no attribute 'database'

dbt.log

[0m13:25:57.121134 [error] [Thread-1  ]: Unhandled error while executing 
'str' object has no attribute 'database'
13:25:57.125079 [debug] [Thread-1  ]: Traceback (most recent call last):
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\task\base.py", line 368, in safe_run
    result = self.compile_and_execute(manifest, ctx)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\task\base.py", line 303, in compile_and_execute
    ctx.node = self.compile(manifest)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\task\compile.py", line 42, in compile
    return self.compiler.compile_node(self.node, manifest, {})
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\compilation.py", line 533, in compile_node
    node = self._compile_code(node, manifest, extra_context)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\compilation.py", line 421, in _compile_code
    node.compiled_code = jinja.get_rendered(
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\clients\jinja.py", line 146, in get_rendered
    rendered = render_template(template, ctx, node)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt_common\clients\jinja.py", line 539, in render_template
    return template.render(ctx)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt_common\clients\jinja.py", line 144, in render
    return self.environment.handle_exception()
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\sandbox.py", line 394, in call
    return __context.call(__obj, *args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\clients\jinja.py", line 84, in __call__
    return self.call_macro(*args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt_common\clients\jinja.py", line 298, in call_macro
    return macro(*args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\runtime.py", line 782, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 2, in template
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\sandbox.py", line 394, in call
    return __context.call(__obj, *args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\clients\jinja.py", line 84, in __call__
    return self.call_macro(*args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt_common\clients\jinja.py", line 298, in call_macro
    return macro(*args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\runtime.py", line 782, in _invoke
    rv = self._func(*arguments)
  File "<template>", line 5, in template
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\jinja2\sandbox.py", line 394, in call
    return __context.call(__obj, *args, **kwargs)
  File "c:\users\user_xxx\repository\dbt_workspace\dbt_repo\venv\lib\site-packages\dbt\adapters\bigquery\impl.py", line 226, in get_columns_in_relation
    database=relation.database, schema=relation.schema, identifier=relation.identifier
AttributeError: 'str' object has no attribute 'database'

Environment

- OS: Windows 10
- Python: 3.9 
- dbt: 1.8.4
- dbt-expectations: 0.10.3
- dbt-bigquery: 1.8.2

Which database adapter are you using with dbt?

dbt-bigquery: 1.8.2

Note: dbt-expectations currently does not support database adapters other than the ones listed below.

Additional Context

Same code added to field under models works properly.

models:
  - name: stg_model
        columns:
          - name: vat_ratio
            description: ""
            tests:
              - dbt_expectations.expect_column_values_to_be_of_type:
                  column_type: float
                  # config:
                  #   severity: warn
                  #   warn_if: "!=0"
dominikszewczyk commented 3 weeks ago

Any idea what can be a problem?