search

calogica / dbt-expectations

Port(ish) of Great Expectations to dbt test macros
https://calogica.github.io/dbt-expectations/
Apache License 2.0
989 stars 120 forks source link

[Feature Request] Add suport for AWS Athena #287

Closed joaopedro02 closed 9 months ago

joaopedro02 commented 9 months ago

Is your feature request related to a problem? Please describe. I need to run tests on tables using AWS Athena as backend. This would enable us to test a great amount of data on a data lake using dbt features.

Describe the solution you'd like Enable the expectations to be run against an AWS Athena connection.

Describe alternatives you've considered I tried to use athena_utils with compatibility "shims" but it does not work with all expectations. And it seems that its not possible to make all expectations compatible in this way because some of then does not use a adapter.dispatch . This turns harder to overwrite some of the dbt package tests code.

Additional context Some expectations that does not work with athena_utils "shims" package: -expect_column_to_exist -expect_row_values_to_have_recent_data

clausherther commented 9 months ago

Hi @joaopedro02, we currently have no way to test against AWS Athena during our CI process, so we can't support this platform at this time.